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ABSTRACT 


The  purpose  of  this  thesis  is  to  trace  and  analyze  the 
evolution  of  Reliability  and  Maintainability  as  engineering 
disciplines.  Articles  published  in  the  open  literature  were 
used  as  the  measurement  indicator  for  developing  the  growth 
curves  of  the  various  branches  within  each  discipline.  The 
growth  curves  were  analyzed  to  determine  the  present 
emphasis  and  to  project  future  trends  within  each  discipline. 
Analyses  were  conducted  to  determine  the  relative  contribu- 
tions made  to  the  growth  patterns  by  private  and  public 
organizations  such  as  the  Department  of  Defense,  Service 
Industries,  etc.  Elements  of  each  discipline  which  indicate 
probable  future  developments  have  been  identified.  Where 
possible,  the  factors  contributing  to  future  growth  have 
also  been  identified.  Taxonomies  have  been  developed  which 
provide  a structured  classification  system  for  the  various 
elements  within  each  discipline.  The  authors  believe  that 
the  taxonomies,  in  conjunction  with  the  growth  curves, 
present  a comprehensible  analysis  of  the  evolution  of  the 
Reliability  and  Maintainability  Disciplines. 


TABLE  OF  CONTENTS 


I.  INTRODUCTION 10 

A.  BACKGROUND 10 

B.  PURPOSE  OF  THESIS  10 

C.  METHOD  OF  RESEARCH 11 

II.  EVOLUTION  OF  RELIABILITY  AND  MAINTAINABILITY  12 

A.  OVERVIEW  OF  THE  ENERGENCE  OF  RELIABILITY 

AND  MAINTAINABILITY 12 

B.  GROWTH  OF  RELIABILITY  AND  MAINTAINABILITY 

BY  THE  DECADES  14 

1.  The  Years  Before  1940  14 

2.  1940  - 1950  15 

3.  1950  - 1960  15 

4.  1960  - 1970  18 

5.  1970  - 1976  19 

III.  METHODOLOGY 21 

A.  INTRODUCTION 21 

B.  METHOD  OF  MEASUREMENT  OF  DISCIPLINE 

DEVELOPMENT 21 

1.  Discipline  Structure  Development  22 

2.  Keyword  Definition  Consistency  24 

C.  TAXONOMY  CONSTRUCTION  25 

D.  DATA  SOURCES  31 

E.  TAXONOMY  VALIDATION  36 

F.  DATA  STORAGE  AND  RETRIEVAL  37 

IV.  ANALYSIS  OF  RELIABILITY  AND  MAINTAINABILITY 

DISCIPLINE  EVOLUTION  39 

A.  INTRODUCTION  39 

B.  RELIABILITY  DISCIPLINE  GROWTH  39 

5 


C.  RELIABILITY  INTRADISCIPLINE  GROWTH  41 

1.  Reliability  Branch  Growth  Trends  42 

2.  Analysis  Branch  - - - 45 

3.  Management  Branch 50 

4.  Test  and  Evaluation  Branch - 58 

5.  Design  Branch - 61 

6.  Theory  Branch 63 

7.  Data  Branch 65 

8.  Reliability  Branch  Correlations  with 

Specific  Applications  65 

9.  Reliability  Branch  Correlations  with 

General  Applications  71 

D.  MAINTAINABILITY  DISCIPLINE  GROWTH  75 

E.  MAINTAINABILITY  INTRADISCIPLINE  GROWTH  77 

1.  Maintainability  Branch  Growth  Trends  77 

2.  Analysis  Branch 79 

3.  Design  Branch - 79 

4.  Test  and  Evaluation  Branch 81 

5.  Data  Branch 84 

6.  Management  Branch - - 86 

7.  Maintainability  Branch  Correlations 

with  General  Applications  89 

F.  RELIABILITY  AND  MAINTAINABILITY  GROWTH 

COMPARISON  94 

V.  CONCLUSIONS  AND  RECOMMENDATIONS  97 

A.  CONCLUSIONS  9 7 

B.  RECOMMENDATIONS  FOR  FUTURE  EFFORT  100 

APPENDIX  A - TAXONOMY  KEYWORD  GLOSSARY  101 

LIST  OF  REFERENCES  10  7 

INITIAL  DISTRIBUTION  LIST  110 


6 


LIST  OF  FIGURES 


* 

1.  Emphasis  on  R and  M from  Books  13 

2.  Reliability  Taxonomy - 26 

3.  Maintainability  Taxonomy - 27 

4.  R and  M Disciplines  Described  by  Tree  Analogy  30 

5.  Literature  Sources - 33 

6.  R and  M Data  Bases 34 

7.  R Discipline  Emphasis - 40 

8.  R Branches  43 

9.  R Analysis  Branch  - 46 

10.  Configuration  Subbranch - 48 

11.  Prediction  Subbranch  — 49 

N 

12.  R Management  Branch  Structure  51 

13.  Management  Branch 52 

14.  System  Effectiveness  Subbranch - 54 

15.  R Program  Management  Subbranch - 55 

16.  Evaluation  and  Assessment  Subbranch  56 

17.  Procurement  Subbranch - 57 

18.  R Test  and  Valuation  Branch 59 

19.  R Statistics  Subbranch  - - 60 

20.  R Design  Branch 62 

21.  R Theory  Branch  - - 64 

22.  R Data  Branch - 66 

23.  Correlation  of  Design  and  Analysis  with 

Specific  Applications 68 

24.  Correlation  of  T 6 E and  Management  with 

Specific  Applications 69 


7 


25. 


— r 


Correlation  of  Analysis  and  Design  with 
General  Applications  72 

26.  Correlation  of  Management  and  T § E with 

General  Applications 73 

27.  M Discipline  Emphasis  - - 76 

28.  M Branches  78 

29.  M Analysis  Branch 80 

30.  M Design  Branch - 82 

31.  M Test  and  Evaluation  Branch - 83 

32.  M Data  Branch - 85 

33.  M Management  Branch  Structure - 87 

34.  Management  Branch - 88 

35.  M Organization  and  Management  Subbranch  90 

36.  Correlation  of  Design  and  Analysis  with 

General  Applications  92 

37.  Correlation  of  Management  and  T 8 E with 

General  Applications 93 

38.  Comparison  of  M M Evolutions  95 

NOTE: 

Reliability  and  Maintainability  have  been  abbreviated 
throughout  the  figures  as  R and  M respectively. 


8 


* + 


■»***•*■  * 


ACKNOWLEDGMENTS 


The  authors  wish  to  express  their  gratitude  to 
Mr.  Walt  Kuzmin,  Mr.  William  Marsh,  and  Mr.  Walt  Hardesty 
of  Systems  Consultants,  Inc.  for  their  assistance  in  the 
development  stages  of  this  thesis.  The  authors  are  also 
grateful  to  Mr.  Burt  Batchelor  of  Boeing  Aerospace  and 
Mr.  Clair  Cunningham  of  Ford  Corporation  for  their  time 
and  patience  in  critiquing  the  taxonomies.  Mr.  E.  J.  Nucci 
of  the  Electronic  Industries  Association  provided  source 
material  that  was  difficult  to  find  and  helped  to  make  our 
research  more  complete.  In  addition,  we  express  our  thanks 
to  Mr.  Jeff  Hall  and  Mrs.  Bernadette  Peavy  of  the  Naval 
Postgraduate  School  for  their  assistance  in  writing  and 
modifying  the  computer  programs,  and  to  Mrs.  Mary  Roberson 
of  the  Naval  Postgraduate  School  for  her  assistance  in 
Systems  Operations.  The  authors  also  thank  our  thesis 
advisors.  Professors  Melvin  Kline  and  James  Esary  for  their 
critique  of  this  thesis,  and  finally,  acknowledgment  of 
the  Naval  Weapons  Center,  China  Lake  for  providing  the 
resources  required  to  accomplish  this  effort. 


9 


I.  INTRODUCTION 

A.  BACKGROUND 

The  ability  of  a system  to  be  operationally  ready  when 
demanded  is  a function  of  its  design.  In  the  early  19,40' s, 
the  designer's  primary  goal  was  to  achieve  an  output  that 
would  satisfy  a desired  set  of  specific  performance 
requirements.  Major  shifts  have  evolved  in  this  thinking 
with  the  rapid  advancements  in  technology  since  World  War 
II.  The  emergence  of  supersonic  aircraft,  nuclear  sub- 
marines, moon  landers,  planetary  exploration,  laser  tech- 
nology, and  complex  weapon  systems  has  highlighted  the  need 
for  efficient  and  economic  designs  in  terms  of  system 
effectiveness  and  cost.  Systems  now  are  required  to  be  both 
reliable  and  maintainable  in  order  to  be  cost-effective  over 
their  designed  life. 

B.  PURPOSE  OF  THESIS 

The  primary  purpose  of  this  thesis  is  to  trace  and 
analyze  the  evolution  of  reliability  and  maintainability  as 
engineering  disciplines  as  evidenced  by  the  published 
literature.  Growth  curves  are  analyzed  to  determine  where 
the  emphasis  in  reliability  and  maintainability  has  been 
over  the  past  three  decades,  what  the  emphasis  is  today, 
and,  finally,  to  project  probable  future  discipline  emphasis. 
Secondary  objectives  were  development  of  a classification 
structure  for  each  discipline,  and  development  of  a 
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substantial  data  base  of  reliability  and  maintainability 
documents  easily  retrieval  for  research  purposes. 

C.  METHOD  OF  RESEARCH 

An  extensive  literature  search  was  performed  to  develop 
a factual  analysis  of  the  evolution  of  these  engineering 
disciplines.  Several  field  trips  were  conducted  to  inter- 
view people  who  are  considered  to  be  highly  knowledgeable 
in  reliability  and  maintainability  in  order  to  obtain  their 
perceptions  of  the  evolution  of  these  fields  and  to  solicit 
their  critiques  on  the  proposed  classification  structures. 
Such  contributions  provided  valuable  assistance  in  the 


early  phases  of  the  project. 


II.  EVOLUTION  OF  RELIABILITY  AND  MAINTAINABILITY 


A.  OVERVIEW  OF  THE  EMERGENCE  OF  RELIABILITY  AND  MAINTAINABILITY 
Literature  published  on  reliability,  maintainability,  and 
quality  control  was  researched  to  identify  the  emergence  of 
these  disciplines  and  to  quantify  their  growth  patterns. 

Quality  control  is  included  because  some  of  the  concepts 
applied  in  the  early  reliability  writings  were  developed  in 
the  quality  control  field.  The  Cumulative  Book  Index  was 
initially  used  for  this  purpose  because  it  is  a reasonably 
complete  and  comprehensive  list  of  works  published  in  the 
English  language.  Also,  works  listed  in  this  reference  must 
be  copyrighted  and  must  have  the  majority  of  the  material 
devoted  to  the  subject  under  which  the  work  is  listed. 

These  three  factors  (English  language,  copyright,  majority 
of  work  devoted  to  listed  subject)  made  the  index  an  ideal 
source  for  providing  an  indication  of  the  growth  pattern  of 
the  reliability  and  maintainability  disciplines.  The  Index 
provided  title,  author,  and  number  of  pages  for  each  work 
listed.  Figure  1 shows  the  evolution  of  quality  control, 
reliability,  and  maintainability  as  evidenced  by  books 
published  in  these  disciplines.  The  indicator  for  growth 
used  is  the  number  of  pages  published  in  three  year  intervals 
beginning  in  1944  and  extending  through  1975.  It  should  be 
noted  here  that  any  conclusions  reached  using  these  curves 
should  be  treated  with  care  since  books  have  an  inherent 
time  lag  between  original  manuscript  preparation 
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and  publication  of  up  to  three  years.  These  curves  are 
presented  only  for  the  purpose  of  giving  a qualitative 
feeling  for  the  discipline  growths  rather  than  trying  to 
pinpoint  exact  emergence  of  the  disciplines. 

As  Figure  1 indicates,  books  on  quality  control  peaked 
about  1951  or  1952  and  declined  thereafter  until  1965  when 
quality  control  was  no  longer  listed  as  an  entity  in  the 
Cumulative  Book  Index.  Reliability  began  appearing  in  the 
early  1950's,  and  maintainability  began  appearing  in  the 
early  1960 's.  Research  into  the  periodical  literature  has 
provided  a more  detailed  breakout  of  some  of  the  events 
which  have  affected  the  development  of  these  disciplines. 

B.  GROWTH  OF  RELIABILITY  AND  MAINTAINABILITY  BY  THE  DECADES 
1 . The  Years  Before  1940 

Reliability  principles  were  used  in  1916  by  the 
Western  Electric  Company  which  is  the  manufacturing  unit  of 
the  Bell  System.  With  production  running  at  a rate  of  about 
ten  million  telephones  annually,  and  rising  rapidly,  studies 
were  initiated  to  discover  means  to  produce  trouble-free 
telephone  equipment  for  public  use.  The  Western  Electric 
Company  was  among  the  first  to  realize  that  the  statistical 
sampling  methods  already  being  formulated  for  science  could 
be  applied  to  industrial  processes.  The  Bell  System  under- 
stood that  durability  must  be  a main  goal,  and  that  service 
history  and  optimization  of  the  design  for  maximum  quality 
were  important  factors. 

* 

fr  14 


— *- 
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The  decade  began  with  a national  emergency  in  which 


there  was  an  urgent  need  for  development  of  a method  for  the 
manufacture  of  uniform  high  quality  products.  The  need  for 
greatly  increased  rates  of  production  lent  itself  readily 
to  the  application  of  the  new  statistical  techniques.  Large 
quantity  production  in  the  rapidly  growing  electrical  and 
electronic  industries  led  the  War  Department  to  develop 
standards  for  the  application  of  statistical  methods  to  the 
quality  control  of  materials  and  manufactured  products. 

The  rapid  growth  of  the  electronics  industries  also  brought 
increasing  problems  of  reliability.  Radar  and  other  military 
developments  of  World  War  II  introduced  the  need  for  specific 
consideration  of  reliability. 

In  1946,  the  commercial  airlines  sponsored  field 
studies  of  tube  and  electronic  equipment  failures.  These 
studies  were  performed  by  Aeronautical  Radio,  Inc.,  (ARINC) . 

3.  1950  - 1960 

As  a result  of  its  work  in  developing  reliable  tubes 
for  the  airlines,  ARINC  was  requested  to  make  an  investigation 
of  military  electronic  tube  reliability.  From  1951  to  1954 


the  program  was  concerned  primarily  with  the  evaluation  and 
improvement  of  tube  reliability.  A general  report  summarizing 
the  findings  was  published  in  1954.  The  effects  of  applica- 
tion, environment,  and  operating  and  maintenance  conditions 
were  shown  to  be  so  closely  related  that  the  program  was 
redirected  to  emphasize  system  reliability  as  affected  by 
electron  tubes.  Systems  under  study  expanded  to  radio 


communication  systems,  radar  systems,  and  bombing  and 
navigation  systems. 


Iv 

I 
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The  recognition  by  the  Department  of  Defense  that 
many  parts  other  than  vacuum  tubes  were  causes  for  problems 
led  to  the  formation  of  the  Ad  Hoc  Group  on  Reliability  of 
Electronics  Equipment  (AGREE)  in  1952  y This  group  was 
instrumental  in  initiating  an  increasing  number  of  studies 
in  order  to  add  to  their  knowledge  of  equipment  failures. 

The  AGREE  Committee  was  at  least  partially  responsible  for 
issuance  of  a Department  of  Defense  Directive  entitled 
’’Reliability  of  Electronics  Equipment”  which  required  that 
additional  emphasis  be  placed  on  the  reliability  of  elec- 
tronic equipment. 

In  1950,  the  VITRO  Corporation  undertook  a study  of 
the  reliability  of  Navy  shipboard  electronic  components  and 
equipments.  This  study  resulted  in  the  establishment  of 
the  relative  failure  rates  of  various  parts  and  in  the 
development  of  an  improved  failure  reporting  system.  In  the 
late  50' s many  other  companies  made  contributions  to  the 
improvement  of  military  electronic  equipment  through  field 
studies  carried  out  under  military  contracts.  These  contri- 
butions were  directed  toward  the  measurement  of  equipment 
reliability  and  the  development  of  methods  for  predicting 
the  reliability  of  new  electronic  equipment  while  it  was 
still  in  the  design  stage. 

The  recognition  of  the  equipment  problems  created  a 
sense  of  reliability  awareness  during  this  period.  Mass 
production  of  systems  caused  the  need  for  standardized  tests 
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to  be  used  in  the  factory.  Reliability  standards  had  to  be 
developed.  Standardization  of  parts  and  circuits  were 
stressed.  Parts  improvement  programs  were  initiated  as  the 
quality  of  parts  left  much  to  be  desired.  Component 
specifications  were  now  being  written.  The  critical  importance 
of  reliability  was  recognized  both  in  the  Department  of 
Defense  and  in  industry.  This  importance  was  emphasized  with 
issuance  of  Military  Handbook  217  and  Mil-Stcl-756A  for 
Reliability  Predictions  as  well  as  Mil-Std-785A  for  Reliability 
Program  Plans  and  Mil-Std-781  for  Reliability  Demonstration 
Tests . 

Prior  to  1954,  maintainability  was  not  a defined 
discipline.  Some  commercial  manufacturers  were  incorporating 
features  later  to  be  known  as  maintainability  features  into 
the  design  of  their  products.  An  example  of  this  was  the 
production  of  standardized  rifles  (Ml  and  Carbine)  for  the 
U.  S.  Army  during  World  War  II.  The  U.  S.  Army  required 
that  the  soldier  be  able  to  disassemble  and  assemble  each 
acceptable  rifle  while  blindfolded  under  combat  and  inclement 
weather  conditions  in  mud.  An  additional  requirement  specified 
that  the  rifle  must  perform  satisfactorily  after  such  as 
assembly.  Therefore,  even  though  maintainability  was  not  yet 
a widely  accepted  discipline,  the  Ml  and  Carbine  rifles 
demonstrated  the  feasibility  of  planning  for  maintainability 
in  the  initial  stages  of  program  development. 

U.  S.  Government  publications  concerning  maintaina- 
bility did  not  exist  in  this  period.  Maintainability 
requirements  were  covered  through  specialized  contractual 
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exhibits  and/or  amendments  to  the  contracts.  By  1959, 
formalized  program  specifications  started  to  evolve.  The 
U.  S.  Air  Force  maintainability  requirements  for  aerospace 
systems  and  equipment  (MIL-M- 26512)  was  one  of  the  first. 

4.  1960  - 1970 

This  decade  witnessed  the  rapid  growth  of  the 
maintainability  discipline.  A realization  that  the  best 
design  from  the  reliability  standpoint  may  be  poor  from 
the  standpoint  of  maintainability  created  a new  challenge 
for  the  design  engineer.  The  need  for  maintainability  was 
predicated  on  the  basis  that  no  system  is  totally  reliable; 
i.e.,  the  system  will  eventually  fail  and  therefore  considera- 
tions must  be  made  on  how  to  return  the  system  to  an  opera- 
tional status  as  rapidly,  as  effectively,  and  as  efficiently 
as  possible.  The  awareness  of  the  need  to  consider  reliability 
and  maintainability  as  design  parameters  early  in  system 
development  began  to  evolve. 

The  development  of  integrated  circuits  and  their 
application  during  this  period  increased  the  complexity  and 
sophistication  of  hardware.  Worst  case  and  statistical 
circuit  analyses  were  used  as  design  tools.  Computerized 
failure  history  data  banks  for  use  in  reliability  predictions 
were  developed.  Reliability  testing  using  statistically 
designed  tests  was  introduced  and  became  recognized  as  a valid 
test  method.  The  systems  effectiveness  concept  was  extensively 
explored.  In  1965  the  first  Naval  Material  Support  Establish- 
ment Systems  Performance  Effectiveness  Conference  was  held. 


MIL-STD-470  was  issued  in  1966  as  the  Department  of 
Defense  preferred  specification  for  Maintainability  Programs. 

A separate  "Maintainability  Prediction"  Handbook  (MIL-HDBK-472) 
was  issued  as  a companion  to  the  new  DOD  maintainability 
standard.  With  issuance  of  a standard  for  maintainability 
demonstrations  (MIL-STD-471)  a fairly  complete  set  of 
maintainability  engineering  implementation  tools  now  existed. 

5.  1970  - 1976 

Cost  factors  became  dominant  during  this  period  and 
emphasis  was  given  to  considerations  such  as  life  cycle  cost 
and  design- to-cost.  Support  costs  became  manpower  intensive, 
as  the  average  cost  of  a soldier  doubled  in  these  six  years. 
Despite  a reduction  of  defense  personnel,  operations  and 
maintenance  costs  rose  for  fewer  operational  units,  and  new 
systems  were  being  acquired  at  a slower  rate.  The  extended 
time  in  service  of  the  old  systems  increased  reliability 
and  maintainability  problems  which  in  turn  increased  the 
costs  for  operation  and  maintenance. 

Reliability  methods  and  procedures  developed  in 
earlier  periods  were  refined  and  were  being  extended  into 
consumer,  energy,  and  nuclear  power  areas.  The  airlines 
pioneered  the  MTBF  guarantee  which  requires  that  the  equip- 
ment supplier  guarantee  a stated  mean-time-between-failure 
in  the  operating  environment.  If  the  guarantee  is  not  met 
the  supplier  must  provide  corrective  action  and  provide  spares 
for  use.  This  MTBF  guarantee  has  been  modified  by  the 
military,  and  DOD  is  using  Reliability  Improvement  Warranties 
as  an  incentive  for  contractors  to  design  equipment  to  meet 
specified  reliability. 
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In  today's  atmosphere  of  increased  cost  consciousness, 
there  is  continued  emphasis  on  reliability  and  added  emphasis 
on  maintainability.  The  defense  community  emphasizes  the 
total  cost  of  ownership,  the  largest  component  of  which  is 
operating  and  support  costs.  This  is  leading  to  a search  for 
innovative  approaches  to  improve  equipment  maintainability. 


III.  METHODOLOGY 


A.  INTRODUCTION 

It  is  apparent  that  reliability  and  maintainability  have 
evolved  from  rudimentary  concepts  into  full  scale  scientific 
disciplines  over  the  past  thirty  years.  To  analyze  this 
evolution,  it  was  necessary  to  devise  a means  to  measure 
their  evolution  and  identify  data  sources  applicable  to  the 
measurement  technique.  This  chapter  addresses  the  measure- 
ment technique,  data  sources,  and  validation  procedures 
utilized. 

B.  METHOD  OF  MEASUREMENT  OF  DISCIPLINE  DEVELOPMENT 

It  was  desirable  to  employ  a measure  that  would  give  as 
accurate  a representation  as  possible  across  the  full 
spectrum  of  both  disciplines.  Dimensions  which  were  con- 
sidered in  the  selection  of  a measurement  method  included 
the  impact  of  reliability  and  maintainability  on  industrial 
products  both  from  user  and  producer  viewpoints,  growth 
measurement  of  such  areas  within  the  disciplines,  applications, 
and  available  data  base.  It  was  considered  important  that 
the  growth  measure  selected  be  broad  enough  to  span  all  of 
the  above  considerations  without  a significant  risk  of  data 
skewing. 

Several  measures  of  discipline  development  were'  con- 
sidered and  the  measure  ultimately  chosen  was  the  number  of 
articles  published  in  the  open  literature  because  this 
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measure  provided  the  broadest  coverage  of  the  disciplines 
with  the  least  amount  of  bias.  Other  measures,  such  as 
government  specifications  would  tend  to  place  undue  emphasis 
on  weapons  systems  and/or  space  related  systems  as  opposed 
to  consumer  capital  items  or  construction  equipment.  Focusing 
on  commercial,  industrial  related  measures  would  have  de- 
emphasized  reliability  and  maintainability  impacts  derived 
from  the  rapid  state  of  the  art  advances  in  technology 
resulting  from  defense  and  space  activities.  Industries 
(particularly  commercial  products)  tend  to  make  small  incre- 
mental changes  in  their  product  technology  levels  whereas 
DOD  and  NASA  tend  to  make  quantum  jumps.  In  one  way  or 
another  all  of  the  alternatives  bias  the  data.  However,  it 
appeared  that  tracking  the  open  literature,  such  as  annual 
reliability  and  maintainability  symposia,  on  a yearly  basis 
would  provide  the  broadest  indicator  of  discipline  development. 

1 . Discipline  Structure  Development 

Once  the  method  for  measuring  discipline  development 
had  been  selected,  it  was  necessary  to  address  questions 
concerning  the  breakdown  of  each  discipline  into  subelements. 
Initially  it  was  unclear  how  to  subdivide  them  and  to  what 
depth.  The  disciplines  could,  for  example,  be  divided  into 
functional  and  application  oriented  subelements  or  they 
could  be  subdivided  by  the  physical  and  mathematical 
sciences  forming  the  core  of  underlying  theory.  All  of  these 
subdivision  alternatives  had  merit,  and  it  was  decided  to 
incorporate  them  into  a hierarchical  classification  system. 

As  a result,  a taxonomy  was  developed  whereby  articles  could 
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be  classified  and  the  data  stored  for  future  analysis  as 
well  as  providing  a mechanism  for  article  retrieval  for 
research  purposes.  The  taxonomy  provided  an  excellent 
structure  for  analyzing  the  development  of  these  disciplines 
along  the  dimensions  mentioned  earlier.  It  was  particularly 
useful  in  defining  the  main  branches  and  emerging  subbranches 
of  the  scientific  core  of  the  disciplines. 

The  keywords  were  initially  selected  by  researching 
a representative  sample  of  the  available  literature  and 
through  successive  refinements  were  finally  arranged  into  a 
classification  structure.  The  final  determination  of  key- 
word location  in  the  taxonomy  and  the  phrasing  of  keyword 
definitions  was  not  a trivial  effort.  Consider,  for  example, 
whether  prediction  should  merit  its  own  elemental  category, 
be  included  with  statistics  as  a subelement  under  quanti- 
tative methods,  or  be  included  as  a subelement  under  analysis. 
At  one  time  or  another  in  the  development  of  the  taxonomy, 
prediction  was  classified  as  all  three.  Eventually,  it  was 
agreed  that  prediction  was  most  frequently  utilized  in 
practice  as  an  analysis  technique  and  it  was  positioned  in 
the  taxonomy  accordingly.  Similar  iterations  have  been 
performed  on  most  of  the  main  branches  of  each  discipline  in 
arriving  at  the  final  structural  relationships. 

The  structure  was  then  presented  to  several  persons 
with  extensive  experience  in  the  fields  of  reliability  and 
maintainability  for  comments.  This  process  was  iterated 
several  times  and  resulted  in  the  taxonomies  presented  in 
Figure  2 (Reliability)  and  Figure  3 (Maintainability) . 
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Along  with  the  determination  of  an  appropriate 


classification  structure,  it  was  particularly  important  to 
maintain  consistency  in  the  manner  in  which  articles  were 
classified  in  the  structure.  For  consistency,  it  was 
necessary  for  everyone  involved  in  the  study  to  have  a clear 
understanding  of  keyword  definitions  within  the  context  of 
discipline  usage.  There  did  not  appear  to  be  a universally 
accepted  glossary  of  reliability  and  maintainability  terms 
available  from  the  literature.  Where  possible,  generally 
accepted  definitions  were  used.  However,  in  some  cases  it 
was  necessary  to  develop  definitions  either  because  none 
existed  or  because  there  were  conflicting  usages  in  the 
literature.  Considerable  effort  was  expended  early  in  the 
study  to  converge  on  a set  of  keyword  definitions  and  to 
solicit  comments  on  the  proposed  definitions  from  experienced 
leaders  in  their  respective  disciplines.  The  definitions 
finally  arrived  at  are  included  in  Appendix  A. 

Each  article  classified  was  studied  to  determine  the 
author's  intended  thrust  and  appropriate  keywords  were  chosen 
to  describe  the  primary  thrust  at  each  level  in  the  hierarchy. 
If  secondary  objectives  were  identified,  these  also  were 
classified.  However,  it  was  considered  important  to  determine 
the  primary  thrust  first  as  this  would  insure  that  the  articles 
were  read  in  sufficient  depth  to  minimize  classification 
errors.  The  main  keyword  classification  system  presently  in 
use  in  the  literature  has  evolved  from  the  ASQC  system  over 
a period  of  years  without  a great  deal  of  planning  either 
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with  regard  to  future  expansion  or  consistency  with  past 
evolution.  It  was  also  apparent  that  authors  had  classified 
their  own  works , and  in  many  cases  this  led  (in  our  opinion) 
to  a substantial  portion  of  the  articles  being  classified 
with  inappropriate  keywords. 

Attempts  were  made  to  use  the  keywords  already 
assigned  to  certain  articles  but  this  proved  difficult  and 
even  misleading.  In  a few  cases  it  was  apparent  that  the 
author  had  attempted  to  expand  the  scope  of  his  article  by 
inappropriately  expanding  the  keyword  designators.  It  was 
felt  that  the  lack  of  generally  accepted  keyword  definitions 
combined  with  the  need  for  a clearly  defined  and  documented 
structural  relationship  of  the  keywords  within  the  disci- 
plines greatly  contributed  to  the  article  author's 
classification  problems.  Recognition  of  this  situation  and 
desire  for  measurement  consistency  contributed  to  the 
taxonomies  depicted  in  Figure  2 and  Figure  3. 

C.  TAXONOMY  CONSTRUCTION 

The  taxonomies  are  arranged  in  a hierarchical  order 
for  the  two  engineering  disciplines.  The  top 
three  levels,  functions,  applications  (general),  and  appli- 
cations (specific)  were  established  to  enable  discipline 
growth  to  be  measured  along  these  dimensions  as  well  as 
within  the  branches  or  elements  of  the  discipline.  In 

t t 

terms  of  growth  measurement,  it  appeared  worthwhile  to 
provide  a means  for  separating  government-oriented  applica- 
tions such  as  defense  and  space  systems  from  industrial 
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FIGURE  2 RELIABILITY  TAXONOMY 


FIGURE  3 MAINTAINABILITY  TAXONOMY 
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applications  such  as  process  control.  As  shown  in  Chapter  IV, 
it  is  interesting  to  correlate  various  functional  activity 
levels  with  elemental  or  branch  developments.  An  example 
of  this  might  be  correlation  of  Department  of  Defense  in- 
fluence on  analysis  of  reliability  in  electronics  systems. 

The  levels  of  ultimate  interest,  of  course,  are  the  elements 
and  subelemental  development  trends  in  both  disciplines. 

Tracing  of  the  development  of  these  lower  levels  chronolog- 
ically provides  a great  deal  of  insight  into  how  the  disci- 
plines evolved  to  their  present  status,  and  correlation  with 
the  upper  levels  provides  an  insight  into  the  underlying 
factors  influencing  their  development.  A secondary  objective 
can  be  realized  by  retaining  these  upper  levels  because  they 
greatly  aid  in  document  identification  and  retrieval  for 
research  purposes. 

As  indicated  in  Figure  2 and  Figure  3,  the  keyword 
structures  for  reliability  and  maintainability  have  a great 
deal  of  similarity  particularly  at  the  upper  levels.  This 
is  not  accidental  nor  is  it  inappropriate  if  one  considers 
that  the  disciplines  are  heavily  interdependent  in  terms  of 
both  application  and  functional  dimensions.  Emphasis  and 
hence  growth  stimulus  have  varied  greatly  within  the 
functional  categories  over  the  years.  For  example,  govern- 
ment agencies  such  as  the  National  Aeronautical  and  Space 
Administration  (NASA)  have  been  extremely  interested  in 
reliability  whereas  the  Department  of  Defense  (DOD)  has 
emphasized  both  reliability  and  maintainability  in  a more 
balanced  sense. 
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In  attempting  to  visualize  the  development  of  these 
disciplines,  it  is  helpful  to  compare  their  development  to 
that  of  a tree  as  pictorially  illustrated  in  Figure  4.  One 
might  represent  the  roots  (which  supply  nutrients  for  growth) 
as  the  sciences  of  mathematics,  chemistry,  physics.  These 
supply  the  technology  and  innovation  for  discipline  develop- 
ment. The  trunk  represents  the  core  of  the  disciplines 
such  as  theory  of  failure  and  theory  of  repair.  The  branches 
represent  growth  of  elements  and  subelements.  As  the  tree 
grows  certain  branches  exhibit  high  growth  rates  and  then 
tend  to  stabilize  or  even  stagnate  and  die.  These  same 
characteristics  are  exhibited  in  both  the  reliability  and 
maintainability  discipline  developments.  Thus,  the  analogy 
is  helpful  in  visualizing  the  growth  patterns. 

In  drawing  this  analogy  it  is  noted  that  maintainability 
has  developed  along  similar  lines  to  reliability  and  in  many 
ways,  was  an  outgrowth  of  reliability.  It  should  be  noted 
that  basic  differences  between  the  two  disciplines  are 
primarily  due  to  different  underlying  concepts  and  theory. 

In  the  analysis  which  follows  in  Chapter  IV,  reliability 
and  maintainability  are  treated  as  separate  entities,  and 
growth  of  branches  and  subbranches  within  each  discipline  is 
measured  and  analyzed.  The  similarities  and  differences 
referred  to  in  the  preceding  discussion  become  more  apparent 
as  the  measurement  and  analysis  of  their  development  from  a 
chronological  standpoint  is  discussed  in  detail. 
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D.  DATA  SOURCES 


In  attempting  to  quantify  the  growth  of  the  various 
branches  within  disciplines,  it  was  decided  to  emphasize 
articles  published  in  the  open  literature  as  opposed  to 
books  because  articles  were  much  narrower  and  more  specific 
in  scope.  Books  tend  to  be  tutorial  in  nature,  and  in 
general,  are  not  representative  of  the  taxonomy.  It  was 
necessary  to  examine  and  classify  a large  number  of  articles 
in  order  to  develop  a data  base  which  was  large  enough  to 
be  significant  and  which,  in  the  aggregate,  would  have 
minimum  bias.  There  is,  of  course,  a much  larger  data  base 
available  and  the  challenge  was  to  select  those  sources 
which  would  be  most  suitable  for  the  purposes  of  this  thesis. 
It  was  desirable  to  include  articles  from  sources  which  were 
somewhat  continuous  in  nature  and  which,  in  the  aggregate, 
covered  the  broad  spectrum  of  both  disciplines.  It  was 
important  to  ensure  that  a concentration  of  articles 
covering  a narrow  spectrum  of  the  disciplines  were  not  in- 
corporated in  any  given  time  interval.  Otherwise,  conclusions 
based  on  the  sample  results  would  not  be  representative  of 
the  total  population.  In  this  regard,  reliability  and  main- 
tainability symposia  proceedings  were  selected  as  the 
primary  sources  for  articles  since  they  are  continuous  in 
nature  and  tend  to  cover  a broad  spectrum  of  topics.  Several 
different  symposia  were  chosen  since  each  one  tends  to  em- 
phasize different  branches  of  the  disciplines,  and  together 
they  appear  to  provide  a well-rounded  coverage  of  each 
discipline . 
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Time  phasing  of  these  sources  and  their  respective  mergers, 
terminations,  etc.,  is  shown  in  Figure  5.  For  reasons  of 
simplicity,  as  symposia  and  proceeding  titles  changed  fre- 
quently, the  most  recent  symposia  proceedings  titles  were 
used.  A complete  list  of  article  sources  chosen  for 
analysis  of  discipline  growth  is  presented  in  the  List  of 
References.  The  reference  numbers  in  the  figure  correspond 
to  the  identifier  numbers  in  the  List  of  References.  As 
indicated  by  Figure  5,  the  sources  selected  span  the  time 
period  of  interest  with  reasonable  consistency  and,  in 
aggregate,  represent  approximately  4,000  articles.  Two 
seemingly  reasonable  sources  were  not  included  in  the  data 
base.  The  Proceedings  of  the  West  Coast  Reliability  Symposium 
were  not  available.  The  Reliability  Abstracts  and  Technical 
Reviews  published  by  NASA  between  1960  and  1970,  did  not 
appear  to  have  reliability  as  a major  thrust  (abstracts  of 
various  reliability  symposia  excepted).  In  addition,  they 
were  of  such  brevity  to  make  classification  to  the  desired 
depth  extremely  difficult,  thus  greatly  increasing  the  risk 
of  erroneous  classification. 


Figure  6 indicates  the  relative  contribution  each  of 
the  sources  made  to  the  overall  data  base  used  for  intra- 


discipline growth  analysis.  The  data  base  for  reliability 
was  reasonably  continuous  and  uniform  in  density.  The  data 
base  for  maintainability,  however,  was  very  spotty  with 
conferences  lasting  only  a few  years  before  termination. 
Maintainability  is  hard  to  find  in  the  open  literature  after 
1970. 
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It  was  decided  that  the  Proceedings  of  the  Annual 
Reliability  and  Maintainability  Symposium  would  be  an 
excellent  base  for  developing  the  taxonomy.  These  pro- 
ceedings started  in  1954,  and,  with  the  exception  of  1955, 
have  been  published  annually  since.  The  reasons  for  this 
decision  were  multiple: 

1.  The  source  was  readily  available. 

2.  The  proceedings  present  a continuous  and  long  time 
span  of  articles  in  the  reliability  discipline. 

3.  The  proceedings  are  a representative  and  compre- 
hensive collection  of  articles  encompassing  the  range  of 
skills  within  the  disciplines. 

From  initial  readings  of  the  proceedings,  it  became 
apparent  that  the  taxonomy  for  reliability  could  readily  be 
constructed  from  the  article  content. 

The  taxonomy  for  maintainability,  however,  could  not  be 
constructed  from  this  source  because  of  an  insufficient 
quantity  of  articles.  The  proceedings  were  not  officially 
designated  a maintainability  source  until  1972  when  the  title 
changed  from  Proceedings  of  the  Annual  Symposium  on  Reliability 
to  Proceedings  of  the  Annual  Reliability  and  Maintainability 
Symposium.  This  occurred  as  a result  of  the  merger  of  the 
Aerospace  Reliability  and  Maintainability  Symposium  with  the 
Annual  Symposium  on  Reliability  as  indicated  by  the  arrow  in 
Figure  5.  Other  sources,  therefore,  were  investigated  for 
development  of  the  taxonomy  for  maintainability. 

The  Maintainability  Committee  of  the  Electronics  Industries 
Association  (EIA)  compiled  two  volumes  of  a maintainability 
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bibliography,  with  the  first  covering  the  period  to  1964  and 
the  second  covering  the  period  1964-1968.  These  two  volumes 
represent  the  first  decade  of  active  development  of  the 
maintainability  discipline.  A close  scrutiny  of  this 
reference  was  made  in  an  attempt  to  locate  a source  or  sources 
that  would  offer  the  same  advantages  for  research  as  the 
Annual  Reliability  and  Maintainability  Symposium  Proceedings. 
An  initial  source  selected  was  the  Proceedings  of  the  NMSE 
Systems  Performance  Effectiveness  Conference  (SPECON)  which 
was  in  existence  from  1965  to  1969.  This  source  was  pri- 
marily concerned  with  qualitative  aspects  and  was  deficient 
with  regard  to  the  quantitative  aspects  of  maintainability. 

A second  source  was  selected,  Proceedings  of  the  EIA 
Conference  on  Maintainability  of  Electronic  Equipment,  which 
covered  the  years  1957  to  1963.  Now,  with  the  Annual 
Reliability  and  Maintainability  Symposium  Proceedings,  an 
overview  of  Maintainability  existed  for  the  period  1957  to 
1976.  It  is  from  these  sources  that  the  taxonomy  for 
Maintainability  was  established. 

E.  TAXONOMY  VALIDATION 

Throughout  the  development  of  the  taxonomy,  the  classi- 
fication scheme  was  subjected  to  a series  of  checks,  primarily 
relating  to  the  naturalness  of  the  keyword  groupings  and 
their  relation  to  the  scientific  basis  for  the  underlying 
theory.  The  initial  groupings  were  taken  from  the  American 
Society  of  Quality  Control  (ASOC)  classification  system  and 
were  subsequently  refined  and  modified  as  more  insight  was 
gained  through  research  of  the  literature.  As  the  taxonomies 


36 


evolved,  many  ambiguities  and  inconsistencies  surfaced  which 
required  resolution.  This  was  accomplished  by  conversations 
with  experienced  practitioners  in  the  fields  of  reliability 
and  maintainability  and  by  a trial  run  consisting  of  classi- 
fying a large  number  of  articles  to  expose  the  broad 
spectrum  of  subbranches  within  the  disciplines.  As  ex- 
perience and  depth  of  knowledge  about  the  disciplines  increased 
through  exposure  to  the  literature,  it  became  progressively 
easier  to  resolve  the  ambiguities. 

In  a less  formal,  yet  equally  meaningful  sense,  the 
taxonomy  was  validated  when  articles  began  to  be  classified 
with  relative  ease.  Finally,  a classification  scheme  was 
developed  that  corresponded  to  the  opinions  of  the  practi- 
tioners and  which  appeared  to  fit  the  patterns  established 
by  an  analysis  of  the  literature  content. 

F.  DATA  STORAGE  AND  RETRIEVAL 

It  was  apparent  from  the  beginning  of  the  thesis  research 
that,  because  of  the  large  number  of  articles  to  be  read  and 
classified,  there  would  be  a monumental  task  associated  with 
the  storage,  control,  and  manipulation  of  the  data.  To 
maintain  control,  each  article  was  assigned  a series  of 
codes  which  served  to  distinguish  it  from  all  others.  An 
IBM  360/67  computer  was  used  to  store  and  manipulate  the 
generated  data.  This  allowed  for  data  storage  by  article 
title,  author,  date  of  publication,  publisher,  keyword,  and 
combinations  thereof.  Specific  information  regarding  the 
assignment  of  keyword  codes,  data  update  procedures  and 
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related  computer  programs  is  included  in  a User's  Manual 
written  as  a companion  document  to  the  thesis.1  Each 
article  entered  into  the  data  bank  was  analyzed  for  content, 
and  classified  by  keyword  according  to  the  taxonomy  developed 
and  presented  earlier.  This  data  (number  of  articles  by 
keywords)  was  then  recalled,  totaled  by  year  for  each  key- 
word, and  plotted  in  the  form  of  a histogram.  The  histograms 
provided  considerable  insight  into  significant  changes  in 
emphasis  which  occurred  in  the  time  period  of  interest.  The 
data  was  then  summed  and  plotted  in  a cumulative  form  to 
gain  a perspective  of  the  overall  growth  characteristics  of 
each  branch.  This  also  provided  a good  indication  of  branches 
which  had  matured  or  which  were  in  the  process  of  maturing. 


^Masten,  R.  L. , Hamilton,  T.  A.,  and  DiPasquale,  J.  A., 
User's  Manual  for  "A  Study  of  the  Evolution  of  the  Reliability 
and  Maintainability  Engineering  Disciplines"  Computer  Programs, 
Naval  Postgraduate  School,  1977. 
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IV.  ANALYSIS  OF  RELIABILITY  AND  MAINTAINABILITY 
DISCIPLINE  EVOLUTION 

A.  INTRODUCTION 

This  chapter  addresses  the  evolution  of  the  reliability 
and  maintainability  disciplines  from  a chronological  stand- 
point. The  measurement  technique  discussed  in  Chapter  III 
is  used  to  analyze  the  development  of  the  disciplines  along 
the  branches  and  subbranches  depicted  in  the  taxonomies. 

Each  discipline  is  analyzed  as  a separate  entity  in  terms 
of  its  internal  development.  Similarities  and  differences 
in  the  development  of  the  two  disciplines  are  noted  in  the 
concluding  section  of  this  chapter. 

B.  RELIABILITY  DISCIPLINE  GROWTH 

The  overall  trend  of  the  reliability  discipline  is 
discussed  before  analyzing  its  intradisciplinary  growth. 
Figure  1 presented  in  Chapter  II  provides  one  measure  of  the 
overall  discipline  growth  using  number  of  pages  published 
in  book  form  as  the  measure  of  growth.  Books,  however,  do 
not  present  the  whole  picture  because  of  the  time  lags 
associated  with  publishing  large  works  and  because  of  the 
wealth  of  literature  published  in  short  article  form.  To 
gain  a perspective  of  the  overall  discipline  growth  a 
composite  picture  was  formed  using  both  books  and  short 
articles.  Figure  7 presents  this  composite  overview,  and 
again  the  unit  of  measure  is  number  of  pages  published  in  a 
three  year  interval.  This  unit  of  measure  is  necessary  in 
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order  to  directly  compare  books  and  articles,  and  it  also 
provides  a smoothing  function  so  that  the  long  term  trend 
is  discernible. 

Figure  7 indicates  that  interest  in  reliability  gradually 
increased  from  the  early  1950 's  to  about  1960  and  then 
dramatically  increased  until  about  1970.  After  1970  there 
appears  to  be  a gradual  decline  in  growth.  Reasons  underlying 
this  phenomenon  can  only  be  conjectured  at  this  point.  The 
growth  during  the  1960 's  can  be  largely  attributed  to  the 
intense  interest  of  the  National  Aeronautics  and  Space 
Agency  (NASA)  and  the  Department  of  Defense  (DOD)  and  the 
accompanying  financial  resources  which  stimulated  the 
aerospace  industry  in  this  time  period.  The  decline  noted 
during  the  early  1970 's  could  be  the  sign  of  a maturing 
discipline,  or  it  could  be  due  to  economic  factors,  since 
both  space  and  defense  have  suffered  budget  cut-backs 
(especially  in  research  and  development)  in  this  time  frame. 
The  decline  is  probably  a combination  of  the  above  factors. 
However,  the  economic  constraints  are  likely  to  be  pre- 
dominant because  technological  advances  which  allow 
development  of  increasingly  complex  systems  stimulate 
technical  innovation  from  reliability  engineers. 

C.  RELIABILITY  INTRADISCIPLINE  GROWTH 

The  method  of  analysis  utilized  to  determine  the  growth 
characteristics  of  the  branches  and  subbranches  of  each 
discipline  has  been  discussed  in  Chapter  III.  For  the 
periodical  literature,  the  measurement  indicator  is  number 
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of  articles  published  per  year,  and  is  plotted  in  the  form 
of  cumulative  histograms  for  each  of  the  branches  and  twigs 
identified  by  keywords.  The  branch,  subbranch,  and  twig  (us- 
ing the  tree  analogy)  growth  patterns  will  be  investigated 
prior  to  correlating  these  elements  with  the  upper  divisions 
in  the  taxonomy. 

1 . Reliability  Branch  Growth  Trends 

Analysis  of  the  main  branches  of  the  taxonomy  (theory, 
design,  etc.)  was  performed  utilizing  the  cumulative  number 
of  articles  published  during  the  time  interval  of  interest. 
This  is  a measure  of  the  interest  and  resources  which  have 
been  applied  to  the  various  branches  of  the  discipline  over 
the  past  twenty- five  years,  and  represents  the  accumulation 
of  knowledge  within  each  branch.  This  data  is  graphically 
presented  in  Figure  8 for  the  period  from  1950  to  1976. 

The  amplitudes  of  the  curves  provide  a good  indica- 
tion of  the  relative  attention  paid  to  the  various  branches. 
The  curves  indicate  that  management,  analysis,  test  and 
evaluation,  and  design  received  far  more  emphasis  than  did 
theory  and  data.  The  slope  of  the  curves  indicates  the 
emphasis  that  each  branch  received  at  a particular  instant 
in  time. 

Theory  and  data  (which  includes  systems  and  tech- 
niques for  collection,  processing  and  utilization  of  the  data) 
appear  to  have  small  and  reasonably  constant  growth  except 
for  a slight  increase  in  emphasis  in  1975  and  1976.  The 
other  four  branches  indicate  a substantial  increase  in 
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emphasis  in  the  1960's  and  1970's.  Analysis  exhibited  the 
most  dramatic  change,  primarily  due  to  the  influence  of 
NASA  and  DOD. 

The  starting  points  for  the  various  curves  show 
when  the  literature  began  to  emphasize  each  branch,  and 
do  not  imply  that  there  was  nothing  written  previously. 
Management  emerged  early  in  the  late  1940's  and  early  1950's, 
providing  the  transition  from  the  quality  control  era  into 
the  reliability  era.  This  phenomenon  was  also  noted  in  books 
as  those  works  slowly  transitioned  from  emphasis  on  quan- 
titive aspects  such  as  statistical  sampling  theory  to 
qualitive  aspects  of  process  control,  and  management  structures 
for  quality  control.  Theory  apparently  emerged  (late  1950's) 
shortly  after  design  and  analysis.  This  seems  reasonable, 
since  in  the  process  of  design  for  fewer  failures,  and  in 
the  analysis  of  failures,  engineers  and  scientists  probed 
deeper  into  the  underlying  physical  causes  of  failure  in 
devices . 

These  curves  provide  an  overview  of  the  growth 
patterns  of  the  main  branches  of  the  discipline  and  it 
appears  that  none  of  them  have  reached  maturity.  Analysis 
and  Management  show  strong  tendencies  toward  increased 
emphasis  in  the  immediate  future.  Within  each  main  branch 
there  are  several  subbranches  which  also  have  pronounced 
growth  patterns  and  these  are  discussed  in  the  following 


2 . Analysis  Branch 

The  literature  indicates  that  the  Analysis  branch 
can  be  split  into  three  subbranches:  (1)  Configuration, 

which  consists  of  those  analytical  techniques  dealing  with 
the  physical  composition  of  systems  of  subsystems.  (2)  Pre- 
diction, and  (3)  Reliability  growth.  The  cumulative  growth 
patterns  exhibited  in  Figure  9 indicate  the  relative  weights 
given  the  subtopics  in  the  literature.  It  is  noted  that 
configuration  which  includes  several  analysis  techniques 
such  as  modeling,  Failure  Modes  Effects  Analysis  (FMEA) , 

Fault  Trees,  and  Reliability  Block  Diagrams  emerged  in  the 
mid  to  late  1950's.  It  received  a lcrge  growth  stimulus  in 
the  early  1960's  and  then  settled  into  a fairly  stable 
growth  pattern  which  has  continued  until  the  present.  The 
early  stimulus  was  primarily  due  to  interest  from  DOD  and 
NASA,  with  the  power  (energy)  and  the  service  industries 
picking  up  the  interest  in  the  1970's. 

Prediction  has  exhibited  characteristics  similar 
to  configuration  but  with  somewhat  lower  emphasis.  It  too 
appears  to  have  received  increased  emphasis  in  the  mid  1970’s 
primarily  from  DOD  and  the  service  industry. 

Reliability  growth  has  not  shown  significant  develop- 
ment in  comparison  to  the  other  subbranches.  There  has  been 
a small  and  relatively  consistent  number  of  articles  each 
year  which  have  addressed  this  subject.  Most  of  the  articles 
seem  to  be  concerned  with  trying  to  show  mathematically 
whether  a product  will  achieve,  or  has  in  fact  achieved,  a 
specified  reliability  level. 
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Both  the  Configuration  and  Prediction  subbranches 
have  been  further  subdivided  into  twigs.  The  twigs  which 
have  grown  out  of  Configuration  are  illustrated  in  Figure 
10.  Modeling/Simulation  and  Failure  Analysis  experienced 
an  extremely  sharp  increase  in  growth  rate  in  1962  and 
this  appears  to  be  primarily  due  to  interest  from  DOD.  The 
other  areas  appear  to  have  experienced  a reasonably  con- 
sistent growth  pattern.  However,  Design  Review  as  an 
analysis  technique  appears  to  have  matured  in  the  late 
1960's.  Fault  Tree  Analysis  appears  to  be  receiving  in- 
creased emphasis  in  the  1970's.  This  could  be  caused  by 
the  growing  interest  in  safety,  stimulated  by  the  electrical 
power  industry.  The  technique  is  presently  one  of  the  strong 
bridges  between  reliability  and  safety  analysis. 

The  literature  indicates  Prediction  split  into  two 
major  areas:  Parameter  Prediction  (estimation)  and 

Probability  Distributions.  Each  of  these  twigs  received 
impetus  about  1960  (see  Figure  11)  which  resulted  in  an 
increased  growth  rate.  Parameter  Prediction  has  sustained 
the  increased  growth  rate  occurring  in  the  early  1960 's. 
However,  there  is  indication  of  a decrease  in  growth  of 
Probability  Distributions  after  1965  in  comparison  with  the 
other  twig.  Maturity  has  not  yet  occurred,  although  the 
trend  indicates  that  it  is  probably  not  far  off. 

The  third  area  under  Prediction,  Apportionment, 
emerged  in  1961,  flourished  for  several  years,  and  then 
appears  to  have  died  off  in  the  late  1960's. 
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This  branch  is  very  broad  in  scope  and  is  the  most 
complex  to  discuss  since  numerous  subbranches  and  twigs  have 
emerged  since  the  1950's.  Figure  12  depicts  the  structural 
development  of  this  branch  using  the  tree  analogy. 

Figure  12  is  intended  to  display  the  structural 
relationships  of  the  Management  branch  without  trying  to 
introduce  the  time  dimension  into  the  illustration. 

The  major  subbranches  of  Management  are  illustrated 
in  Figure  13.  All  of  the  subbranches  and  twigs,  with  the 
exception  of  Product  Liability,  have  received  major  impetus 
to  their  growth  from  DOD  and  NASA. 

The  Reliability  Program  Management,  Cost,  Failure 
Recurrence  Control,  and  Procurement  subbranches  all  emerged 
very  early  in  the  literature.  As  noted  in  Figure  13, 
Reliability  Program  Management  has  received  far  more  emphasis 
than  any  of  the  other  subbranches.  This  subbranch  received 
a major  impetus  from  DOD  in  the  early  1960 's  and  the  in- 
creased growth  pattern  has  carried  on  into  the  early  1970' s. 
There  are  indications  that  it  may  be  approaching  maturity 
because  of  the  economic  cut-backs  mentioned  in  Chapter  II. 

The  other  main  subbranches,  Failure  Recurrence  Control, 
Procurement,  and  Cost  appear  to  be  exhibiting  fairly  con- 
sistent growth  patterns.  Failure  Recurrence  Control  (efforts 
by  Management  to  curtail  and/or  preclude  recurrence  of 
reliability  failures)  is  indicating  signs  of  approaching 
maturity  in  the  mid-1970's,  whereas  both  Procurement  and 
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Cost  are  showing  signs  of  increasing  emphasis.  This  is 
especially  true  of  the  twig,  Systems  Effectiveness,  shown 
in  Figure  14. 

Systems  Effectiveness  emerged  in  the  early  1960's 
and  has  established  and  maintained  a very  healthy  growth 
pattern  to  the  present.  This  area  has  been  primarily 
driven  by  DOD,  and  it  is  expected  that  this  emphasis  will 
continue,  at  least  in  the  foreseeable  future.  Product 
Liability  emerged  in  the  early  1970 's  and  is  giving  every 
sign  of  growing  into  a major  area  in  the  near  future. 

The  main  subbranch,  Reliability  Program  Management, 
has  developed  several  twigs,  and  their  growth  patterns  are 
illustrated  in  Figure  15.  All  of  these  areas  have  sustained 
fairly  consistent  growth  patterns  with  the  exception  of 
Training/Education  which  appears  to  have  reached  maturity 
in  the  mid  1 970's.  Field  Activity  emerged  in  the  mid-1960's 
and  has  only  recently  experienced  some  growth. 

Evaluation  and  Assessment  can  be  further  split  into 
three  twigs  as  shown  in  Figure  16.  It  can  be  noted  that 
all  three  of  these  areas  have  received  added  emphasis  in 
the  early  1970's. 

Procurement,  the  remaining  subbranch  of  Management 
to  be  discussed,  has  generated  three  major  twigs  as  illus- 
trated in  Figure  17.  The  growth  patterns  for  specifications 
and  contracts  exhibit  somewhat  similar  characteristics  with 
both  having  received  stimulus  in  the  mid  1960 's  and  much 
less  emphasis  in  the  early  1970' s.  Data  from  the  last  couple 
of  years,  however,  indicates  renewed  emphasis  in  these  areas 
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primarily  from  DOD  and  the  service  (utilities,  airlines, 
etc.)  industries.  Warranties  emerged  in  the  late  1950's 
and  continues  to  exhibit  a pattern  of  increased  growth. 

4 . Test  and  Evaluation  Branch 

The  Test  and  Evaluation  (T  § E)  branch,  illustrated 
in  Figure  18,  is  split  into  three  main  subbranches: 

Methods  (of  testing),  Statistics,  and  Reporting/Evaluation. 
There  appears  to  have  been  greater  emphasis  given  to  Methods 
than  either  of  the  other  two  subbranches.  This  emphasis 
seems  to  have  occurred  in  the  early  1960 's  and  the  resulting 
growth  pattern  has  continued  on  into  the  present.  The  T 8 E 
branch  emerged  in  the  earliest  literature  and  appeared  to 
be  an  extension  of  the  Quality  Control  (QC)  discipline.  The 
literature  gradually  transitioned  from  Quality  Control 
related  issues  into  Reliability  during  the  1950's.  The 
combined  influence  of  DOD  and  NASA  appear  to  be  the  major 
motivation  for  the  rapid  increase  noted  for  Methods  of 
Testing  during  the  early  1960 's. 

Statistics  was  definitely  a carry  over  from  Quality 
Control  and  development  of  several  twigs  in  this  subbranch 
are  noted  in  the  literature.  These  are  illustrated  in 
Figure  19.  Sampling  Plans  and  Design  of  Experiments  are 
the  most  direct  carry  overs  from  Quality  Control  and  it 
appears  that  Design  of  Experiments  has  matured  in  the  mid 
1970's.  Bayesian  techniques  and  Parameter  Estimation  have 
exhibited  the  most  active  growth  patterns.  Parameter 
Estimation  had  exhibited  a relatively  mild  growth  rate  until 
1970,  at  which  point  it  appears  to  have  received  added 
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emphasis.  This  seems  to  correlate  with  the  increased 
attention  from  the  service  industries,  especially  the  Power 
Industry,  to  testing  in  general. 
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Bayesian  techniques  emerged  about  1963  and  exhibited 
a rapidly  increasing  growth  pattern  until  about  1972  when 
it  dramatically  decreased,  but  seems  to  be  picking  up  again. 

The  reasons  underlying  this  are  not  identified  at  this 
point.  The  other  twigs,  Confidence  Level  and  Goodness  of 
Fit,  have  exhibited  low  growth  characteristics,  *and  there 
is  insufficient  data  available  to  comfortably  project  future 
activity  in  these  areas. 

5 . Design  Branch 

This  branch  has  produced  several  twigs  which,  for 
classification  purposes,  have  been  treated  as  separate 
entities.  Design  has  been  split  into  component  related 
activities  and  engineering  analysis  activities  to  separate 
the  curves  for  ease  and  clarity  of  data  presentation.  This 

II 

data  is  presented  in  Figure  20.  From  the  upper  portion  of 
Figure  20,  it  can  be  noted  that  Part/Material  Selection  and 
Component  Testing  have  been  the  most  active  areas  in  this 
branch.  Both  of  these  twigs  exhibited  dramatic  increases 
in  growth  starting  about  1964,  and  the  increased  growth  rate 
has  continued  to  the  present.  The  sharp  increase  in  growth 
appears  to  have  been  initiated  by  interest  from  NASA  and 
later  sustained  by  both  DOD  and  NASA.  The  other  two  areas. 

Redundancy  and  High  Reliability  Parts,  have  exhibited  much 
lower  growth  patterns.  In  fact,  High  Reliability  Parts 
appears  to  have  reached  maturity.  Redundancy  appears  to 
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be  receiving  increased  emphasis  in  the  mid  1970's  and  this 
seems  to  be  primarily  due  to  articles  concerned  with 
reliability  in  the  Mars  Probe  and  Space  Shuttle  (space 
applications) . 

The  other  twigs  which  have  developed  from  the  Design 
Branch  are  presented  in  the  lower  portion  of  Figure  20. 
Criteria/Requirements  and  Tolerance/Safety  factors  have 
exhibited  reasonably  consistent  growth  patterns  since  they 
emerged  in  the  mid  1950's.  Tolerance/Safety  factors  re- 
ceived a surge  of  interest  in  the  mid  1960 's  but  soon 
settled  back  into  its  previously  established  growth  pattern. 
Test  equipment  emerged  in  1960  and  appears  to  have  received 
consistent  emphasis  until  the  mid  1970 's  at  which  point  it 
seems  to  have  matured.  Perhaps  it  is  being  presented  in 
other  formats.  There  has  been  relatively  consistent  dis- 
cussion in  the  literature  of  burn-in  since  about  1965  to  the 
present. 

6 . Theory  Branch 

This  branch  discernibly  emerged  in  the  literature 
in  the  mid  to  late  1950's.  The  primary  focus  appears  to  be 
Theory  of  Failure  and  this  subbranch  has  exhibited  a 
relatively  constant  growth  pattern  since  the  sharp  increase 
in  slope  in  about  1961,  (see  Figure  21).  Renewal  Theory  as 
applied  in  this  context,  has  emerged  as  a twig  in  the  early 
1960 's  and  has  exhibited  a fairly  consistent  growth  pattern 
to -the  present.  Renewal  theory,  as  treated  in  the  literature, 
is  another  of  the  bridges  between  reliability  and  maintain- 
ability. 


63 


7.  Data  Branch 


The  Data  Branch  has  received  the  least  emphasis  of 
all  the  main  branches  of  the  reliability  discipline. 

However,  it  has  demonstrated  a reasonably  consistent  growth 
pattern,  as  shown  in  Figure  22.  This  branch  has  produced 
two  major  twigs,  Data  Collection  and  Data  Processing/ 

Utilization.  Both  twigs  emerged  very  early  in  the  1950 's 
and  exhibited  a fairly  constant  growth  pattern  until  1969 
or  1970  at  which  point  the  growth  curves  show  an  increasing 
slope.  This  trend  seems  to  be  resulting  from  increased 
emphasis  by  both  DOD  and  the  service  industries  on  the  sub- 
ject of  making  effective  use  of  data  for  both  reliability 
, and  maintainability  analysis,  prediction,  allocation,  and 
spare  parts  determination. 

8 . Reliability  Branch  Correlation  with  Specific  Applications 
Articles  which  could  be  classified  along  the  applica- 
tion dimension  as  well  as  along  the  reliability  intradisci- 
pline dimensions  have  been  retained  in  the  computer  stored 
data  base  and  provisions  were  made  to  retrieve  these  articles 
by  requesting  the  appropriate  combination  of  keywords.  This 
has  provided  a good  mechanism  for  correlating  the  growth 
of  the  more  active  branches  with  specific  applications  of 
the  techniques  contained  within  the  branches.  Only  the  four 
most  active  branches  (Design,  Analysis,  Management,  and 
Test)  have  been  selected  for  correlation  because  these  re- 
flect the  major  emphasis  of  the  reliability  discipline  and, 
hence,  contain  sufficient  data  to  establish  some  meaningful 
trends.  Figure  23  presents  the  correlations  of  Design  and 
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Analysis  with  specific  applications.  Figure  24  presents 
Management  and  Test  methods  also  correlated  with  specific 
applications . 
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In  Figure  23,  the  curves  correlating  Design  with 
specific  applications  indicate  some  rather  dramatic  changes 
in  emphasis  of  this  branch  over  the  last  two  decades. 

Design  applied  to  Space  Transportation  has  received  more 
emphasis  than  any  other  application  during  the  1960 's  and 
then  it  appears  to  have  matured  in  the  1970's.  Air 
Transportation  design  applications  were  mentioned  in  the 
earliest  literature,  and  it  has  displayed  a relatively 
constant  growth  rate  until  the  mid  1970's  when  it  seemed 
to  approach  maturity.  Communication  and  Medical  Applica- 
tions also  seem  to  be  approaching  maturity  in  the  mid  1970's 
after  having  experienced  fairly  heavy  emphasis  in  the  late 
1960's  and  early  1970's.  Computer  Applications  (including 
both  software  and  hardware)  emerged  in  the  literature  in 
the  late  1950 's  and  has  experienced  increasing  emphasis 
ever  since.  Ground  Transportation  (primarily  rapid  transit) 
emerged  in  the  mid  1960 's  and  is  experiencing  increasing 
emphasis  in  the  mid  1970' s.  Power  Generation  (includes  both 
nuclear  and  conventional)  emerged  in  the  early  1970 's  and  is 
showing  signs  of  phenomenal  growth  during  the  1970 's  if 
the  present  trend  continues. 

The  lower  set  of  curves  in  Figure  23  indicates  the 
relative  emphasis  which  Analysis  has  had  in  various  applica- 
tions. The  same  general  trends  are  evident  in  this  branch 
as  were  noted  for  the  Design  branch.  However,  it  appears 
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Figure  23  Correlation  of  Design  and  Analysis  with  Specific  Applications 
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that  Analysis  has  lagged  Design  by  two  or  three  years. 

There  does  not  appear  to  be  any  Analysis  applications  that 
have  matured,  although  Space  and  Air  Transportation  are 
exhibiting  signs  that  this  may  happen  in  the  not  too  dis- 
tant future.  Analysis  applications  emphasis,  like  Design, 
appears  to  have  shifted  to  Computer  and  Power  Generation 
in  the  1970's.  Both  of  these  application  areas  have  re- 
ceived very  high  growth  stimulation  in  the  last  few  years 
and  there  is  every  reason  to  believe  that  these  trends 
will  continue,  at  least  in  the  near  future. 

Figure  24  presents  the  Test  § Evaluation  and 
Management  branches  correlated  with  the  same  specific 
application  areas  discussed  earlier  for  the  Design  and 
Analysis  branches.  Some  of  the  trends  established  for 
Analysis  and  Design  are  also  evident  in  Figure  24.  For 
example,  Space  Transportation  applications  has  clearly 
reached  maturity  in  the  early  1970's  for  both  T § E and 
Management.  Other  applications  presented  in  the  T § E 
portion  of  Figure  24  all  indicate  stable  growth  patterns  in 
the  1970 's.  T § E application  in  the  Power  Generation 
field  emerged  in  the  mid  1970's  and  is  in  such  an  infant 
state  that  trend  prediction  will  not  be  attempted  now 
except  that,  it  is  likely  that  it  will  experience  signifi- 
cant growth  because  of  the  increase  in  the  other  branches. 

Management  applications  other  than  Space 
Transportation  and  Communications  appear  to  be  exhibiting 
strong,  stable  growth  patterns.  Ground  Transportation 
emerged  in  the  mid  1960 's  and  has  achieved  a strong  growth 


70 


pattern  (most  of  the  emphasis  here  has  been  in  rapid  transit 
considerations  for  cities).  Power  Generation  has  again 
taken  off  with  every  indication  of  establishing  a very  strong 
growth  pattern. 

Management  applications  in  the  Medical  Field  emerged 
in  the  late  1960's  and  appears  to  have  established  a stable 
growth  pattern.  A large  part  of  the  emphasis  here  has 
dealt  with  the  effect  of  medical  device  failures  on  patient 
safety. 

9 . Reliability  Branch  Correlations  with  General  Applications 

Throughout  the  preceding  discussions  of  the  evolu- 
tion of  the  reliability  discipline,  several  statements  have 
been  made  concerning  factors  and  organizations  which  have 
provided  major  stimulus  to  reliability  growth.  Data  which 
support  these  statements  are  presented  in  Figure  25  and 
Figure  26.  These  data  are  presented  in  the  form  of  densi- 
ties instead  of  the  cumulative  format,  in  order  to  gain  an 
insight  into  the  emphasis  that  particular  organizations 
have  placed  on  the  major  Reliability  branches  at  any  given 
point  in  time. 

Figure  25  indicates  the  correlation  of  the  Analysis 
and  Design  branches  with  selected  general  application 
categories.  The  top  curve  (double  thickness)  in  each  plot 
represents  the  overall  density  of  articles  which  addressed 
Analysis  or  Design  regardless  of  application.  The  cross 
hatched  areas  under  these  curves  represent  the  emphasis 
placed  by  various  organizations  at  various  points  in  time. 

For  example,  in  the  Analysis  portion  of  Figure  25,  it  can 
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be  noted  that  DOD's  emphasis  on  Analysis  peaked  in  the 
mid  1960's  and  declined  thereafter  until  the  mid  1970's. 
Service  industries  (which  include  airlines,  utilities, 
telephone  companies,  etc.)  began  emphasizing  Analysis  in 
the  mid  1960's  and  have  progressively  increased  their 
interest  to  the  present.  ERDA  began  emphasizing  Analysis 
in  the  early  1970's,  also  with  steadily  increased  emphasis. 
The  trends  noted  for  Analysis  are  typical  of  those  es- 
tablished by  the  other  main  branches.  There  are  minor 
shifts  in  time  when  various  branches  were  emphasized  by 
either  DOD  or  NASA,  however  the  general  trends  appear 
consistent  for  the  four  branches  analyzed  in  Figures  25 
and  26. 

Correlations  of  these  branches  were  also  run  with 
the  other  general  application  areas  indicated  in  the 
taxonomy.  There  was  very  low  activity  in  all  of  the  other 
areas  with  the  exception  of  consumer  capital  items.  This 
application  has  received  steadily  increasing  emphasis  in 
both  analysis  and  design  activity  for  the  last  two  years. 

The  literature  surveyed  does  indicate,  though,  that  DOD 
applications  are  beginning  to  place  renewed  emphasis  on 
Management  in  the  mid  1970 's.  The  literature  indicates 
that  this  is  probably  due  to  increased  awareness  of  the 
magnitude  of  their  Operations  and  Maintenance  Budget  and 
the  impact  that  reliability  of  equipment  in  the  field  has 
on  this  budgetary  item.  These  same  considerations  have 
also  provided  a good  deal  of  impetus  to  the  growth  of  the 
maintainability  discipline  as  described  in  the  next  section. 
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D.  MAINTAINABILITY  DISCIPLINE  GROWTH 


A composite  picture  using  both  books  and  short  articles 
was  formed  in  order  to  gain  a perspective  of  the  overall 
discipline  growth.  Figure  27  represents  this  composite 
overview.  Again,  the  unit  of  measure  for  this  comparison 
is  the  number  of  pages  published  in  a three  year  interval. 

Figure  27  indicates  that  interest  in  maintainability 
began  about  1957,  sharply  increased  to  about  1966,  and 
then  dramatically  decreased  until  about  1972.  The  decline 
noted  in  the  1970 's  could  be  the  beginning  sign  of  a 
maturing  discipline  or  it  could  be  due  to  a shift  in  in- 
terest away  from  maintainability  towards  greater  emphasis 
on  reliability.  The  growth  in  complexity  of  systems  led 

to  the  realization  that  equipment  reliability  could  not  be 

/ 

improved  to  the  extent  that  the  need  for  maintenance  could 
be  economically  eliminated.  Although  the  search  for 
further  reliability  improvement  continued,  it  became  unwise 
to  plan  future  systems  without  considering  maintainability 
in  the  design.  The  significant  increases  in  operating  and 
support  costs  provided  additional  support  for  the  rising 
interest  in  maintainability.  This  increase  in  costs  occurred 
while  developing  agencies  were  working  with  increasingly 
limited  funds.  The  overall  growth  of  Maintainability  was 
strongly  influenced  by  recognition  of  this  engineering 
discipline  as  a critical  element  in  system  effectiveness  by 
the  military. 
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E.  MAINTAINABILITY  INTRADISCIPLINE  GROWTH 


This  analysis  was  conducted  by  plotting  the  cumulative 
number  of  articles  published  on  selected  topics  of  main- 
tainability as  described  earlier. 

1 . Maintainability  Branch  Growth  Trends 

The  curves  in  Figure  28  indicate  that  management 
has  received  far  more  emphasis  than  any  of  the  other 
branches.  This  emphasis  in  management  stems  from  the  im- 
position of  firm  military  standards  on  all  DOD  contracts 
beginning  in  the  late  1960's.  Analysis  and  Design  emerged 
in  the  1950's  when  maintainability  criteria  were  being 
developed.  Their  growth  then  increased  as  maintainability 
requirements  became  better  defined  in  the  early  1960 's. 
Analysis  and  Design  seem  to  have  matured  in  the  1970 's. 

Data  growth  rose  in  the  early  1960 's,  began 
flattening  out  in  the  late  1960 's  and  appears  to  have 
matured  in  the  1970 's.  The  rise  appears  to  be  primarily 
due  to  the  military  standard  that  directs  the  contractor 
to  establish  data  collection,  analysis,  and  corrective 
action  systems. 

t Maintainability  Test  § Evaluation  emerged  in  the 

late  1950's,  rose  slowly  in  the  middle  1960 's  and  then  has 

i 

increased  to  the  present.  The  growth  increase  noted  in  the 
late  1960 's  appears  to  be  derived  from  the  military  re- 
quiring maintainability  demonstration  tests  by  the  contractor. 
Growth  tendencies  within  each  of  the  above  main  branches  are 

I 

developed  in  the  following  sections. 

* 

V 
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2 . Analysis  Branch 

The  literature  indicates  that  the  analysis  branch 
can  be  split  into  Simulation/Modeling,  Prediction,  and 
Maintenance  Engineering  Analysis  (MEA) . The  growth 
patterns  exhibited  in  Figure  29  indicate  the  emphasis 
given  the  subbranches  in  the  literature.  It  can  be  noted 
that  prediction  received  a large  growth  stimulus  in  the 
early  1960's  and  then  slowed  down.  This  stimulus  was 
primarily  due  to  interest  from  the  military  services  in 
developing  methods  of  quantifying  measures  of  maintaina- 
bility. Simulation/Modeling  has  exhibited  the  largest 
growth  in  this  branch  as  engineers  attempted  to  develop 
models  to  test  and  evaluate  their  assumptions.  This 
subbranch  growth  increased  from  the  middle  to  late  1960 's 
and  then  appears  to  have  settled  into  a stable  (and 
rapidly  increasing)  growth  pattern  in  the  1970 's.  Main- 
tenance Engineering  Analysis  (MEA)  has  also  shown  a strong 
emphasis  in  the  1960 's  with  a lower,  but  steady  growth  in 
the  1970's.  The  MEA  growth  in  the  1960 's  can  be  linked 
to  the  added  interest  in  Integrated  Logistic  Support  (ILS) 
concepts.  Maintainability  Design  Reviews  as  an  analytical 
tool  seems  to  have  received  some  impetus  in  the  mid- 1960 's 
and  then  matured  in  the  early  1970 's. 

3 . Design  Branch 

This  branch  has  produced  several  twigs  and  Figure  30 
shows  the  relative  emphasis  each  has  received.  Throughout 
the  literature,  human  factors  engineers  have  been  described 
as  those  primarily  responsible  for  the  development  of 
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maintainability  design  guides.  The  curves  indicate  this 
interest  began  in  the  1950 's  and  continued  through  the 
following  decade.  The  declining  growth  in  the  1970 's 
seems  to  reflect  the  maturation  of  ideas  by  the  human 
factors  engineers. 

The  literature  shows  a rising  interest  in  qualita- 
tive attributes  from  1960  through  the  mid-1960's.  The 
growth  leveled  off  in  the  1970 's  indicating  that  qualitative 
design  features  had  matured.* 

Quantitative  maintainability  design  criteria  relate 
to  equipment  features  that  enhance  maintenance  time  re- 
duction. These  requirements  have  been  specified  for  defense 
electronic,  aircraft,  and  missile  systems  for  about  15  years 
The  rapid  growth  of  design  for  quantitative  requirements  is 
graphically  portrayed  in  Figure  30.  The  early  quantitative 
maintainability  specifications,  established  in  1960, 
stimulated  defense  contractors  to  put  greater  emphasis  on 
design  requirements.  The  effort  to  meaningfully  quantify 
requirements  continues  to  the  present  day. 

Trade-off  studies  as  a design  technique  sustained 
a high  growth  rate  in  the  1960's  and  then  appears  to  have 
matured  in  the  early  1970's. 

4 . Test  and  Evaluation  Branch 

The  Test  and  Evaluation  branch  can  be  split  into 
three  main  subbranches:  test  methods,  statistics,  and 

reporting/evaluation  as  shown  in  Figure  31.  Greater 
emphasis  appears  to  have  been  given  to  test  methods  than  to 
either  of  the  other  two  subbranches.  This  emphasis  started 

in  the  early  1960's  and  has  continued  to  the  present. 
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The  literature  clearly  indicates  that  DOD's  interest 
in  maintainability  testing  started  in  the  early  1960's. 

This  interest  has  continued  into  the  1970 's  with  publica- 
tion of  the  updated  Mil-Std-471A  for  maintainability  demon- 
stration in  March,  1973.  This  standard  established  uniform 
procedures,  test  methods,  and  verification  for  evaluation 
of  the  achievement  of  specified  maintainability  requirements. 

A normal  carry-over  from  demonstration  testing  is 

the  need  for  feedback  data  and  other  reporting  mechanisms. 

Therefore,  the  reporting  and  evaluation  subbranch  received 

emphasis  in  1965  and  leveled  off  in  the  early  1970's. 

Additional  growth  has  been  stimulated  in  the  mid- 1970 's. 

* 

The  application  of  statistics  in  developing  test 
methods  has  been  a natural  development.  Maintainability 
demonstration,  for  example,  is  primarily  concerned  with  the 
measurement  of  active  maintenance  downtime,  and  measurement 
of  both  preventive  and  corrective  maintenance  downtime  is 
generally  conducted  as  a statistical  test  under  carefully 
defined  conditions.  Mil-Std-471A  emphasizes  statistical 
techniques  in  its  specified  test  methods,  test  parameters 
(mean,  median,  critical  percentile),  assumed  distributions, 
sample  size,  sample  selection,  and  specification  require- 
ments. Interest  in  statistical  techniques  shows  a steady 
increase  starting  in  1965  and  maturing  about  1970. 

5 . Data  Branch 

Data  collection  and  its  utilization  received 
significant  interest  in  the  1960's  (see  Figure  32).  Efforts 
to  develop  maintainability  data  collection  systems  in 
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response  to  military  requirements  provided  a great  deal  of 
this  impetus.  Data  collection  grew  dramatically  in  the 
1960's  and  appears  to  have  matured  in  the  1970 's  as  the 
military  services  implemented  the  developed  systems.  The 
Navy  3M  system  is  an  example  of  such  a data  system.  Data 
utilization,  however,  continued  some  growth  in  the  1970 's 
and  appears  to  be  growing  still.  This  growth  can  partially 
be  attributed  to  the  increased  uses  to  which  these  data 
are  being  put;  for  example,  quality  assurance  feedback, 
product  improvement  requirements,  and  spare  parts  manage- 
ment. The  literature  also  indicated  interest  exhibited 
in  a technique  called  Design  Disclosure  formats.  This 
method  of  disclosing  design  information  emerged  in  the 
early  1960 's  and  appears  to  have  matured  in  the  1970's. 

6 . Management  Branch 

This  branch  is  the  most  difficult  branch  to  assess 
as  it  has  the  most  complex  structure.  In  order  to  more 
clearly  discuss  this  branch,  it  was  decided  to  illustrate 
the  structural  relations  as  was  done  for  reliability 
management  (Figure  33) . Figure  34  shows  the  five  main 
subbranches,  (1)  Organization  and  Management,  (2)  Logistics, 
(3)  Availability,  (4)  Procurement,  and  (5)  Cost.  Organiza- 
tion and  Management  is  split  into  the  two  main  twigs  of 
Planning  and  Implementation  and  Evaluation  and  Assessment. 

The  Procurement  subbranch  has  developed  two  twigs,  Contracts 
and  Specifications.  All  subbranches  showed  growth  patterns 
in  the  1960  - 1970  decade.  Organization  and  Management's 
growth  can  be  traced  to  the  military's  desire  to  formalize 
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maintainability  into  a firm  requirement.  This  was  done 
when  MIL-STD-470  and  a number  of  supporting  standards  were 
issued.  The  curves  in  Figure  35  indicate  that  Program 
Planning  and  Implementation  appeared  to  grow  into  the 
1970's  and  shows  a tendency  to  level  off.  -Evaluation  and 
Assessment  techniques  seemed  to  grow  for  a short  period 
of  time  and  then  mature  in  the  late  1960 's.  Logistics 
concerns  grew  dramatically  in  the  1960's  as  DOD  put 
increased  emphasis  on  operation  and  support  costs.  Interest 
in  the  amount  of  maintainability  in  Logistics  appeared  to 
decline  in  the  1970's.  This  indicates  that  usage  of  main- 
tainability techniques  in  Logistics  has  matured.  Availability 
does  not  show  any  dramatic  increase  other  than  from  mid 
1960's  to  early  1970's.  This  increase  related  to  the  aware- 
ness that  operational  Availability  deals  not  only  with 
failure  (reliability) , but  also  with  repair  (maintainability) . 

In  this  time  frame,  many  articles  in  the  literature  stressed 
ideas  on  how  to  improve  availability  by  considering  main- 
tainability  in  addition  to  reliability.  However,  availa- 
bility appears  to  have  experienced  little  or  no  growth  in 
the  1970 's  except  in  the  past  two  years.  Procurement's 
rise  through  the  1960's  is  traced  to  specification  of 
maintainability  requirements  in  military  contracts.  The 
need  to  specify  quantitive  requirements  forced  more  consider- 
ation of  maintainability  in  specifications. 

7 . Maintainability  Branch  Correlation  with  General  Applications 
Correlation  of  the  most  active  main  branches  of 
maintainability  with  general  applications  indicates  a strong 
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influence  of  maintainability  interest  within  the  Department 
of  Defense  and  a lesser  influence  by  NASA.  The  data  base 
does  not  show  a large  interest  in  maintainability  in  con- 
sumables, construction,  or  industrial  short-life  equipment. 
The  literature  indicates  a concern  for  maintainability  by 
those  companies  manufacturing  equipments  which  they  sell 
or  lease  but  continue  to  maintain  throughout  the  equipment's 
service  life.  Computer  and  copying  machine  manufacturers 
are  examples.  The  thrust  for  consumables  was  more  towards 
reliability  and  quality  control.  Maintainability  interest 
with  respect  to  industrial  long-life  equipment  appears  to 
have  been  a concern  in  connection  with  commercial  aircraft. 
Here  again,  the  reasons  appeared  to  be  economics  and  cus- 
tomer satisfaction,  since  the  airlines  perform  their  own 
maintenance. 

The  heavy  influence  of  the  Department  of  Defense 
is  dramatically  portrayed  in  Figures  36  and  37.  Its  in- 
terest appeared  in  the  late  1950's,  rose  sharply  through 
the  1960's,  then  appeared  to  decrease  just  as  sharply  in 
the  late  1960 's  and  early  1970 's  to  the  levels  noted  in  the 
1950's  which  it  has  maintained  through  the  present  day. 

NASA  interest  in  maintainability  has  been  primarily 
in  space  and  its  environments.  The  literature  has  been 
sparse  with  respect  to  number  of  published  articles. 

Figures  36  and  37  show  that  awareness  of  maintainability  by 
NASA  did  not  really  take  hold  until  the  middle  1960  's  and 
it  peaked  in  the  late  1960  's  when  the  space  program  design 
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was  at  its  peak.  Maintainability  interest  in  the  1970's 
can  be  expected  to  increase  with  the  skylab  and  space 
shuttle  programs. 

In  conclusion,  the  greatest  thrust  for  maintaina- 
bility has  come  from  the  Department  of  Defense. 

F.  RELIABILITY  AND  MAINTAINABILITY  GROWTH  COMPARISON 

The  preceding  sections  in  this  Chapter  have  attempted 
to  examine  the  growth  of  the  reliability  and  maintainability 
disciplines  by  developing  growth  curves  of  the  various 
branches  within  each  discipline.  The  background  discussions 
in  Chapter  II  provided  some  insight  into  the  events  which 
affected  the  growth  curves  discussed  earlier  in  this  Chapter. 

Figure  38  provides  an  overview  of  the  disciplines  and 
summarizes  how  the  emphasis  has  shifted  within  each  discipline 
over  the  past  fifteen  years. 

Figure  38  indicates  that  reliability  and  maintainability 
main  branch  growths  have  progressed  differently  in  any  given 
time  period.  Reliability  had  very  high  emphasis  on  Manage- 
ment and  Test  8 Evaluation  in  the  late  1950 's  and  early 
1960's.  However,  by  1965  the  emphasis  had  started  shifting 
more  to  Analysis  and  Design  and  this  trend  has  continued 
to  the  present.  Maintainability,  on  the  other  hand,  had 
very  high  emphasis  on  design  in  the  early  1960's,  but 
emphasis  has  subsequently  shifted  to  Management  and  Test  8 
Evaluation. 

These  trends  correlate  well  with  the  events  noted  in 
Chapter  II  where,  for  example,  it  was  noted  that  some  of 
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the  concepts  developed  for  quality  control  were  initially 
applied  to  reliability  considerations  during  *he  1950's. 
Quality  control  has  a very  strong  management  flavor  and 
this  is  reflected  in  the  high  management  emphasis  of 
reliability  illustrated  in  Figure  38  for  1960.  This  area 
and  other  aspects  of  management  have  received  proportionately 
less  emphasis  (with  respect  to  the  other  branches)  in  the 
1970's.  The  late  1950's  and  early  1960's  saw  an  increase 
in  study  groups  and  organizations  (AGREE  and  ARINC,  for 
example)  primarily  devoted  to  analysis  of  systems  and 
components  and  their  influence  is  reflected  in  the  dramatic 
increase  in  emphasis  of  Analysis  (Reliability)  in  the  mid 
1960's.  Design  (Reliability)  has  also  gradually  received 
increased  emphasis  over  the  years,  and  this  has  led  to 
development  of  specification  and  reliability  design  hand- 
books, particularly  for  DOD  and  NASA. 

The  initial  strong  Design  emphasis  in  maintainability 
also  produced  a large  number  of  specifications  and  design 
handbooks.  However,  as  this  discipline  has  grown,  the 
flavor  has  shifted  to  Management  and  Test  6 Evaluation. 

Test  6 Evaluation  (Maintainability) , particularly  in  the 
form  of  demonstrations  to  satisfy  procurement  specifications, 
appears  to  be  steadily  growing  in  emphasis,  and  it  is 
expected  that  this  trend  will  continue. 


V.  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  CONCLUSIONS 

Several  conclusions  regarding  present  and  future  em- 
phasis of  reliability  and  maintainability  have  been  reached 
as  a result  of  the  analysis  of  the  growth  trends  of  the 

disciplines.  As  noted  earlier,  DOD  and  NASA  have  strongly 

# 

influenced  the  development  of  both  Reliability  and  Main- 
tainability. Current  trends,  however,  indicate  that  the 
energy  (power)  and  the  other  service  industries  will  play 
a major  role  in  influencing  the  growth  trends  in  the  future 
particularly  for  Reliability.  Maintainability  growth  will 
probably  continue  to  be  dominated  by  DOD  because  of  its 
growing  concern  with  the  magnitude  of  operations  and  main- 
tenance cost  of  weapon  systems.  There  seems  to  be  growing 
concern  in  DOD  and  the  defense  industry  that  the  present 
specifications  for  reliability  and  maintainability  required 
in  most  weapon  systems  contracts  are  outdated  because  of 
technological  advances  made  in  recent  years.  Reliability 
and  Maintainability  Committees  of  Industry  Associations, 
such  as  EIA,  are  investigating  updating  these  standards  and 
specifications  jointly  with  government  agencies.  Several 
of  the  maintainability  branches,  notably  Theory,  Data,  and 
Design  are  approaching  maturity  since  the  literature  indicates 
almost  no  activity  in  these  areas  in  recent  years. 

Analysis  is  showing  signs  of  continuing  growth,  although 
maturity  may  be  reached  in  the  near  future.  Two  twigs  in  this 


branch,  Simulation/Modeling  and  MEA  are  indicating  strong 
growth  trends  and  it  appears  that  this  will  continue. 

The  Management  and  Test  and  Evaluation  branches  also 
show  signs  of  continuing  growth.  Two  twigs  in  the  Test 
and  Evaluation  branch,  Demonstration  Methods  and  Reporting/ 
Evaluation,  are  also  exhibiting  strong  growth  patterns  at 
present . 

Should  the  maintainability  specifications  be  revised  as 
proposed,  then  the  Management  and  Test  and  Evaluation  branches 
should  receive  renewed  interest. 

The  emphasis  within  the  reliability  discipline  in  recent 
years  has  shifted  from  the  Management  and  Test  and  Evaluation 
branches  to  the  Analysis  and  Design  branches.  Nevertheless, 
all  of  the  main  branches  of  Reliability  are  exhibiting 
healthy  growth  patterns  at  present,  with  Analysis  and 
Management  indicating  increased  activity  in  the  last  two 
years.  This  increased  growth  rate  should  continue  under  the 
impetus  of  renewed  DOD  interest  in  Management,  and  especially 
the  newly  elicited  interest  of  ERDA  and  the  service  industries 
in  both  Management  and  Analysis.  Applications  of  the  tech- 
niques developed  in  the  Analysis  and  Design  branches  to 
computational  and  power  generation  problems  appear  to  be 
fast  growing  areas  at  present.  This  is  particularly  true 
of  Power  Generation  which  has  established  a phenomenal 
growth  rate  in  the  last  three  years  and  progress  in  this 
area  should  be  very  interesting  to  monitor  in  the  future. 
Several  twigs  have  emerged  which  show  promise  of  high  con- 
tinued growth  in  the  future.  These  twigs,  in  conjunction 

with  their  motivational  impetus  are  listed  below. 
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TWIG 

MOTIVATION 

1. 

■\ 

System  Effectiveness 
(cost) 

2. 

Prediction 

3. 

Modeling/Simulation 

j DOD  and  Service 

4. 

Warranties  § 

•> 

Specifications 
5.  Product  Liability 


Consumer/Legal 


The  twig,  Product  Liability,  is  potentially  a very 
complex  area  with  safety  and  reliability  issues  combined 
with  the  complicated  legal  implications  involved  with 
consumer  protection.  This  twig  is  experiencing  a sub- 
stantial growth  rate.  These  issues  are  important  because 
court  decisions  could  have  significant  effects  on  the 
amount  of  Analysis  and  Testing  a manufacturer  may  have  to 
undertake  in  order  to  demonstrate  that  his  product  is  re- 
liable and  to  precisely  define  the  environmental  rating. 

It  appears  that  the  literature  in  recent  years  has  been 
devoting  progressively  more  attention  to  the  implications  of 
equipment  reliability  on  safety  issues.  Analysis  tech- 
niques such  as  Fault  Trees  have  been  developed  which  are 
useful  in  both  reliability  and  safety  analysis.  Many 
articles  have  dealt  with  these  techniques  as  utilized  in 
safety  analysis  as  well  as  the  effect  of  "unreliability" 
on  safety.  This  area  has  been  noticeably  growing  in  the 
recent  past  and  appears  to  have  potential  for  rapid  develop- 
ment in  the  immediate  future. 
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B.  RECOMMENDATIONS  FOR  FUTURE  EFFORT 


1.  Continue  to  update  the  data  base  with  latest  symposia 
proceedings.  This  effort  has  produced  an  extensive  data 
base  for  the  reliability  and  maintainability  disciplines 
classified  by  keywords  along  the  dimensions  discussed  in 
Chapter  III.  This  data  base,  if  kept  current,  represents 

a valuable  source  of  information  which  is  easily  retrievable 
for  research  purposes. 

2.  Publish  a bibliography  on  reliability  and  maintaina- 
bility. The  data  base  on  reliability  and  maintainability 
should  be  made  available  to  other  persons  working  in  the 
disciplines . 

3.  Develop  the  evolution  of  safety  over  the  decades. 

The  safety  discipline  is  closely  interrelated  with  reliability 
and  maintainability  and  if  developed  would  be  a valuable 
source  of  information  to  future  researchers. 

4.  Present  the  taxonomies  to  symposia  and  professional 
societies  for  acceptance  as  standard  classification  systems. 
The  taxonomies  are  composed  of  hierarchical  ordered  key- 
words whose  structural  relationships  have  been  documented. 

The  structural  relationships  are  considered  to  be  superior 
to  those  presently  in  existence. 

5.  Expand  the  data  base  to  include  technical  reports 
and  classified  documents.  This  will  tend  to  fill  in  voids 
left  by  the  current  data  base  and  provide  a more  complete 
analysis  of  the  trends  established  herein. 
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APPENDIX  A 


TAXONOMY  KEYWORD  GLOSSARY 
A. 0.  INTRODUCTION 

This  glossary  of  taxonomy  keywords  is  presented  to 
provide  the  reader  with  the  keyword  definitions  used  by 
the  authors  to  insure  consistency  in  article  keyword 
assignment.  The  glossary  addresses  only  those  taxonomy 
keywords  about  which  there  was  some  definition  disagreement 
amongst  the  authors,  or  which  was  required  for  reader 
clarification.  The  glossary  is  not  intended  to  be  a dic- 
tionary of  reliability  and  maintainability  terms  for 
general  use. 

The  keyword  definitions,  where  possible,  have  been 
taken  from  the  literature  when  it  appeared  that  the 
definition  had  general  usage.  Definitions  were  generated 
for  keywords  that  did  not  have  generally  accepted  definitions 
and  these  definitions  have  been  identified  by  asterisks. 
Keywords  which  have  generally  accepted  definitions  such  as 
Mechanical  or  Communications,  and  keywords  which  are 
taxonomy  headings,  such  as  Management  and  Prediction,  have 
not  been  defined.  Where  needed  for  clarity,  the  taxonomy 
branch  to  which  the  keyword  being  defined  belongs  has  been 
parenthetically  inserted  following  the  keyword. 

A. 1 . DEFINITIONS 

AEC/ERDA  - articles  which  deal  with  energy  either  sponsored 
by  or  written  for  the  Atomic  Energy  Commission  or  the  Energy 
Resource  Development  Agency. 
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*Analysis  and  Evaluation  - (Test  § Evaluation)  process  of 
working  with  the  reported  test  results  to  arrive  at  con- 
clusions relative  to  the  item  tested. 

Apportionment  - an  allocation  of  the  overall  numerical 
reliability  or  maintainability  requirements  among  each  of 
the  elements  of  a system. 

Attribute/Parameter  Estimation  - (Test  § Evaluation)  - 
measurement  and/or  correlation  of  some  characteristic  of 
a part  or  system  by  test  procedures  wherein  the  character- 
istic is  quantitatively  or  qualitatively  estimated  by 
analyzing  the  test  data  to  determine  if  the  part/system 
possesses  the  characteristic. 

Availability  - the  probability  that  the  system  is  operating 
satisfactorily  or  is  ready  to  be  placed  in  operation  at 
any  point  in  time  under  stated  conditions  of  use. 

*Bayesian  - technique  of  modifying  conclusions  reached 
about  present  data  based  on  historical  knowledge  of  the 
data  characteristics. 

*Block  Diagram  - a graphical  method  relating  system  success 
to  component  successes. 

Burn- in  - the  operation  of  an  item,  component  or  system 
to  stabilize  its  characteristics  or  to  weed  out  its  faults 
prior  to  placing  the  item  in  use. 

*Case  Studies  - how  specific  reliability  or  maintainability 
programs  were  approached  and/or  how  specific  program 
problems  were  resolved.  The  essence  of  these  articles  is 
not  analytical  but  rather  lessons  learned  for  discussions 
of  "How  we  did  what." 

*Component  Testing  (Design)  - subjecting  components  or 
systems  to  environments  designed  to  verify  that  they  will 
perform  their  intended  function  in  the  environments  in 
which  they  will  be  utilized. 

^Consumables  - items  such  as  ammunition,  food  and  fuel  which 
are  expended  during  operation  of  the  system. 

*Consumer  Capital  Item  - items  such  as  automobiles  and 
appliances,  which  by  virtue  of  their  cost  and  life  duration 
represent  a significant  portion  of  the  average  citizen's 
budget . 

*Cost  - the  costs,  monetary  or  otherwise,  of  a reliability 
or  maintainability  program,  including  articles  concerning 
major  programs  or  concepts,  such  as  design- to-cost , 
operation  and  maintenance  costs,  and  their  relationship  to 
reliability  and  maintainability. 
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*Criteria/Requirements  (Design)  - goals  and  specified 
performance  values  which  components  and/or  systems  are 
expected  to  achieve  in  their  operational  environment. 

Criticality  Analysis  - a quantitative  method  of  ranking 
the  failure  modes  according  to  the  criticality  of  their 
effects  to  their  probability  of  occurrence. 

*Data  Collection  - methods,  forms,  practices,  etc.,  for 
the  acquisition  and  storage  of  data  concerning  the  per- 
formance of  specified  entities. 

*Data  Exchange  - programs  for  the  exchange  of  data  within 
the  reliability  and  maintainability  communities.  The 
definition  includes  articles  presenting  bibliographies 
developed  to  facilitate  the  exchange. 

*Data  Utilization  - purposes  for  which  information  'con- 
cerning systems  or  component  performance  is  to  be 
collected. 

Design  Disclosure  Format  (Data)  - a method  of  presenting 
critical  maintainability  data  for  the  system  under  design 
in  a manner  that  facilitates  analysis. 

*Design  of  Experiments  (Test  $ Evaluation)  - development 
of  test  procedures/methods  in  such  a way  that  statistical 
principles  may  be  utilized  in  working  with  the  test  data 
and  in  reaching  conclusions. 

*Design  Reviews  (Analysis)  - a comprehensive  critical  audit 
of  all  aspects  of  the  design  of  the  hardware  and  software. 

*Design  Reviews  (Management)  - management  reviews  of 
reliability  and  maintainability  programs.  These  reviews 
tend  to  be  macro  program  reviews  vice  the  more  micro 
design  reviews  identified  under  analysis. 

*DOD  - articles  dealing  with  defense  applications  sponsored 
by  or  written  for  or  about  the  Department  of  Defense. 

*Failure  Recurrence  Control  - articles  concerning  management 
efforts  to  curtail  and/or  preclude  the  recurrence  of 
reliability  failures.  Includes  articles  addressing  re- 
liability failures  which  arise  during  or  as  a result  of 
development  or  production. 

Fault  Tree  - a graphical  method  using  logic  calculus  to 
relate  system  failures  to  component  failures  or  failure 
modes . 

*Field  Activity  - end  user  conditions  caused  by  inadequate 
reliability  or  maintainability,  and  efforts  to  improve  or 
rectify  the  situation. 
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FMEA  - a qualitative  technique  for  analyzing  and  evaluating 
a design  by  singling  out  failure  modes  and  analyzing  the 
consequences  and  causes. 

*General  (Functional  Area)  - articles  which  do  not  address 
a functional  area,  or  articles  where  the  functional  area 
though  addressed  is  immaterial  to  the  essence  of  the 
article. 

^Goodness  of  Fit  - technique  to  determine  if  a group  of 
data  can  be  expected  to  behave  in  accordance  with  a hy- 
pothesized probability  distribution  at  some  predetermined 
level  of  significance. 

*Government  (Other)  - articles  dealing  with  applications 
sponsored  by  government  agencies  other  than  AEC/ERDA, 

NASA,  or  DOD.  This  includes  foreign  governments. 

*High  Reliability  Parts  (Design)  - those  parts  or  compo- 
nents which  by  virtue  of  design,  construction  or  testing 
are  more  resistant  to  failure  than  parts  with  similar 
form,  fit  and  function. 

Human  Engineering  - the  area  of  human  factors  which  applies 
scientific  knowledge  to  the  design  of  items  to  achieve 
effective  man-machine  integration  and  utilization. 

industrial  Long  Life  or  High  Cost  - items  such  as  heavy 
machine  tools  which  require  a significant  portion  of  an 
industrial  concern's  capital  budget. 

industrial  Short  Life  or  Low  Cost  - items,  such  as  test 
equipment,  utilized  for  industrial  purposes  which  represent 
expensed  rather  than  capitalized  expenditures  or  which 
represent  a small  portion  of  an  industrial  concern’s  budget. 

Logistics  - the  procurement,  movement,  maintenance  and 
disposition  of  supplies,  equipment,  facilities  and  personnel, 
and  the  provision  of  services. 

MEA  - the  formal  process  of  reviewing  a design  or  proposed 
design  for  the  purpose  of  identifying  and  quantifying  the 
maintenance  and  support  requirements  for  an  operational 
system. 

Measure  - the  use  of  theory  which  allows  quantification  of 
maintainability  parameters  so  they  can  be  measured. 

Modeling  - conceptualizing  the  interactions  of  the  elements 
comprising  an  entity  in  quantifiable  terms  usually  for  the 
purposes  of  behavior  prediction  and/or  forming  hypotheses 
about  current  or  past  behavior. 
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*NASA  - articles  which  deal  with  space  applications  sponsored 
by  or  written  for  or  about  the  National  Aeronautics  and 
Space  Administration. 

*Non- reversible  - devices  which  depend  upon  some  physical, 
chemical  or  biological  reaction  or  effect  which  once 
started  cannot  be  reversed  or  changed  back  to  its  original 
form  or  state. 

*Parameter  Prediction  - techniques  for  estimating  the 
characteristics  of  a population  based  on  the  knowledge  of 
the  behavior  of  a sample  drawn  from  the  population  or  a 
knowledge  of  its  constituent  parts,  functions  and  operating 
environment . 

*Part/Material  Selection  * selection  of  those  parts  which, 
by  design  or  from  test  results,  are  more  failure  resistant 
than  similar  parts. 

Probability  Distribution  - a mathematical  description  of 
the  relative  frequency  with  which  the  various  outcomes  of 
an  experiment  can  be  expected  to  occur. 

*Product  Liability  - implied  responsibility  of  manufacturer 
to  ensure  that  their  products  perform  their  mission  safely 
and  satisfactorily.  Consequences  of  product  failures  may 
be  found  to  be  responsibility  of  product  manufacturer. 

*Program  Evaluation  (Management)  - assessment  and  interpre- 
tation of  reliability  or  maintainability  programs. 

*Qualitative  Attributes  - those  techniques  such  as  access- 
ibility, test  points,  that  allow  ease  of  maintenance. 

*Quantitative  Attributes  - those  characteristics  that  can 
be  assessed  and  measured  or  demonstrated  during  established 
design,  development,  production,  and  field  use  milestones. 

Redundancy  (Reliability,  Design)  - the  existence  of  more 
than  one  path  for  accomplishing  a given  task,  where  all 
means  must  fail  before  there  is  an  over-all  failure  of 
the  system. 

^Reliability  Growth  - the  process  of  predicting  future 
reliability  achievements  by  extrapolating  measured  data 
on  current  reliability  characteristics. 

Renewal  Theory  - the  study  of  certain  probability  problems 
which  can  arise  in  connection  with  the  failure  and  replace- 
ment of  components. 

Safety  Factor  - the  margin  of  conservativeness  inherent  in 
the  application  of  component  parts  as  a function  of  all 
stresses.  Includes  using  components  in  environments  less 
stressful  then  normally  designed  for  (derating). 
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Sampling/Screening  Plan  - a statement  of  the  sample  size  or 
sizes  to  be  used  and  the  associated  acceptance  and  rejection 
criteria. 

*Service  - articles  sponsored  by  or  written  for  or  about  the 
service  industries,  (ie.  utilities,  airlines,  telephone). 

*Simulation/Modeling  - conceptualizing  the  interactions  of 
the  elements  comprising  an  entity  in  quantifiable  terms, 
usually  for  the  purposes  of  behavior  prediction  and/or 
forming  hypotheses  about  current  or  past  behavior. 

Specifications  (Procurement)  - considerations  related  to 
specifications  or  the  statement  of  work. 

System  Effectiveness  (Cost)  - the  measure  of  the  extent  to 
which  a system  can  be  expected  to  complete  its  assigned 
mission  within  an  established  time  frame  under  stated 
environmental  conditions. 

*Test  Equipment  - equipment  utilized  to  subject  components 
or  systems  to  conditions  designed  to  verify  the  item's 
performance  under  specified  conditions. 

*Testing/Demonstration  Methods  - techniques  to  measure 
whether  systems,  components,  etc.,  will  perform  at  specified 
or  required  levels. 

Tolerance  Analysis  - study  of  the  allowable  variation  in 
values  of  a component  or  system  characteristics  within  which 
the  characteristics  are  judged  acceptable. 

Trade-off  - a process  of  optimization  of  the  related  factors 
involved  during  the  life  cycle  of  an  equipment  or  system 
to  determine  an  optimal  balance  among  these  factors  to 
accomplish  a specified  objective. 

* Unspecified  (Application,  General)  - application  unspecified 
in  the  article . 
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